Method and system for pro-active detection and correction of low quality questions in a question and answer based customer support system

ABSTRACT

User satisfaction with answers that may be provided through a question and answer based customer support system is predicted by pre-submission analysis of the attributes of the question itself before the answer is generated. Questions being entered into the question and answer based customer support system that are not likely to satisfy either an asking user submitting a question, or subsequent searching users accessing the resulting question and answer pair, are labeled improperly formatted questions, or low quality format questions. The question data representing improperly formatted questions is then either filtered out, avoided completely, or, proactively corrected by providing the user with a set of personalized question format transformation instructions to transform improperly formatted questions into a re-phrased/re-formatted properly formatted questions having a question format predicted to provide a significantly higher probability of user satisfaction with any answer eventually provided.

BACKGROUND

Software applications and systems have become indispensable tools forhelping consumers, i.e., users, perform a wide variety of tasks in theirdaily professional and personal lives. Currently, numerous types ofdesktop, web-based, and cloud-based software systems are available tohelp users perform a plethora of tasks ranging from basic computingsystem operations and word processing, to financial management, smallbusiness management, tax preparation, health tracking and healthcaremanagement, as well as other personal and business endeavors,operations, and functions far too numerous to individually delineatehere.

One major, if not determinative, factor in the utility, and ultimatecommercial success, of a given software system of any type is theability to implement and provide a customer support system through whicha given user can obtain assistance and, in particular, get answers toquestions that arise during the installation and operation of thesoftware system. However, providing potentially millions of softwaresystem users specialized advice and answers to their specific questionsis a huge undertaking that can easily, and rapidly, become economicallyinfeasible.

To address this problem, many providers of software systems implement orsponsor one or more question and answer based customer support systems.Typically, a question and answer based customer support system includesa hosted forum through which a user can direct their specific questions,typically in a text format, to a support community that often includesother users and/or professional support personal.

In many cases, once a user's specific question is answered by members ofthe support community through the question and answer based customersupport system, the user's specific question, and the answer to thespecific question provided by the support community, is categorized andadded to a customer support question and answer database associated withthe question and answer based customer support system. In this way,subsequent users of the software system can access the user's specificquestion or topic, and find the answer to the user's question, via asearch of the customer support question and answer database. As aresult, a dynamic customer support question and answer database ofcategorized/indexed user questions and answers is made available tousers of the software system through the question and answer basedcustomer support system.

The development of customer support question and answer databases hasnumerous advantages including a self-help element whereby a searchinguser, i.e., a user accessing the resulting question and answer pair, canfind an answer to their particular question by simply searching thecustomer support question and answer database for topics, questions, andanswers related to their issue. In addition, if the answer to the user'sspecific question is not in the customer support question and answerdatabase, the user can then become an asking user by submitting theirquestion to the question and answer based customer support system,typically through the same web-site and/or user interface. Consequently,using a question and answer based customer support system including acustomer support question and answer database, potentially millions ofuser questions can be answered in an efficient and effective manner, andwith minimal duplicative effort.

Using currently available question and answer based customer supportsystems, once an asking user's question is answered, the asking user isprovided the opportunity to rate the answer with respect to how helpfulthe answer was to the asking user. In addition, searching users in theuser community are provided the opportunity to access the question andanswer data in the customer support question and answer database andthen these searching users are also provided the opportunity to rate theaccessed question and answer content based on how helpful the answer wasto them. In this way, feedback is provided with respect to a givenquestion and answer pair, and answers with low satisfaction ratings,i.e., poorly rated answers, can eventually be identified by thisfeedback. Typically, a poorly rated answer is then eventually removedfrom the customer support question and answer database.

However, using current question and answer based customer supportsystems, and their associated customer support question and answerdatabases, the poorly rated question and answer content is only removedafter it has potentially been viewed by multiple users, and often alarge number of searching users. Consequently, by the time poorly ratedquestion and answer content is identified by a threshold number of lowsatisfaction ratings, such as a “down vote,” of the answer content, notonly is the initial asking user potentially dissatisfied with the answercontent, and often with the software system itself, but additionalsearching users, and often numerous additional searching users, are alsopotentially dissatisfied with the poorly rated question and answercontent, as well as the support provided, and the software systemitself. In addition, these current methods for identifying poorly ratedquestion and answer content are based on the assumption that the userswill not only provide feedback, but that they will provide feedback thatis objective and logical, e.g., not based on emotion or frustration;Often, this is simply not the case.

The fact that, currently, poorly rated question and answer content isonly removed after it has been viewed, and the answer content voted downby multiple users, and often a large number of users, is a significantissue and a long standing problem for question and answer based customersupport systems and software system providers. This is because usersatisfaction with the question and answer based customer support systemsis not only critical to the effectiveness of the question and answerbased customer support systems, but also to the satisfaction andreputation of the software system and the software system provider.Consequently, one of the most significant long standing problemsadversely affecting question and answer based customer support systemsis the inability to ensure user satisfaction with answer contentprovided through the question and answer based customer support systemsin relative real-time, or at least before multiple users are providedaccess to low quality question and answer content. The current lack ofan efficient and effective solution to this problem means that,currently, both users and providers of software systems, and questionand answer based customer support systems of all types, are denied thefull potential of question and answer based customer support systems. Asa result, the technical fields of information dissemination, customersupport, feedback utilization and integration, software implementationand operation, and user experience are detrimentally affected.

What is needed is a method and system for reliably and efficientlypredicting answer quality, and user satisfaction with a potential answerto the user's question, before the answer to the question is generatedand provided to users. In this way, not only can the individual askinguser's satisfaction with an answer be predicted before the answer isprovided to the user, but the satisfaction of other searching users withthe question and answer content can be predicted to ensure answerslikely to result in poor user satisfaction ratings are never provided tothe users.

SUMMARY

Embodiments of the present disclosure address some of the shortcomingsassociated traditional question and answer based customer supportsystems by focusing initial analysis on the question being asked by theuser before any specific answer is considered, generated, or provided toany users. In this way, the question itself is used to predict usersatisfaction with potential answers that may, or may not, eventually beprovided through the question and answer based customer support system,and before any resources are actually devoted to generating andproviding an answer to the question.

In one embodiment, the method and system for pro-active detection andcorrection of low quality questions submitted to a question and answerbased customer support system includes predicting user satisfaction withanswers that may eventually be provided through the question and answerbased customer support system by performing pre-submission analysis ofthe attributes, subject matter, and format of the question itself,rather than post question submission analysis of the answer contentprovided in response to the question. This paradigm shifting approach topredicting user satisfaction with an answer based on the user's questionalone, and before the answer is generated, is in direct contrast toprior assumptions and approaches that focused entirely on analysis ofthe answer content provided after both the question and answer hadalready been formulated.

In one embodiment, using the disclosed method and system for pro-activedetection and correction of low quality questions submitted to aquestion and answer based customer support system, questions beingentered into the question and answer based customer support system thatare not likely to satisfy either an asking user, i.e., a user submittinga question, and/or subsequent searching users, i.e., users accessing theresulting question and answer pair content through a customer supportquestion and answer database, are labeled improperly formattedquestions, i.e., a question having a low quality question format knownto provide a low probability of user satisfaction with any answercontent eventually provided. In one embodiment, the undesirable questioncontent representing improperly formatted questions is either filteredout, avoided completely, or corrected by providing the asking orsearching user with a set of personalized question format transformationinstructions to guide the user and transform an improperly formattedquestion into a re-phrased, and/or modified, properly formatted questionhaving a high quality question format known to provide a significantlyhigher probability of user satisfaction with any answer contenteventually provided.

In one embodiment, the question reformation instructions used totransform an improperly formatted question into a properly formattedquestion are generated by dynamically analyzing the format of the user'squestion as submitted; in one embodiment, as the question is beingformed/created and entered into the question and answer based customersupport system. In short, in one embodiment, as a user is enteringquestion data, the question data is analyzed to identify improperlyformatted questions having low quality question formats. If improperlyformatted questions are identified, the users are provided formattransformation instructions and/or suggestions on how tore-phrase/reform the improperly formatted questions. In one embodiment,the format transformation instructions for transforming the improperlyformatted question into a properly formatted question are customized tothe specific question being submitted, in relative real-time. As aresult, improperly formatted questions having low quality questionformats are transformed into properly formatted questions having a highquality question formats before the question is submitted for response,and before any resources are devoted to actually trying to answer theimproperly formatted question.

As a specific illustrative example, in one embodiment, an asking user'squestion is analyzed as it is being entered into the question and answerbased customer support system and if the question is determined to be animproperly formatted question because the question is of a low qualitybroadly framed general knowledge/open-ended question format, the askinguser is provided format transformation instructions and guided through astep-by-step process to transform the identified generalknowledge/open-ended question format into a properly formatted questionhaving a high quality question format, such as, for example, aclosed-ended question format, capable of being answered with a simple“yes” or “no”, or a closed-ended question format capable of beinganswered via multi-choice, or mapping. In one embodiment, the formattransformation instructions are used to implement this the step-by-steptransformation process before the question is submitted to the questionand answer based customer support system for response, and before anyresources are devoted to actually trying to answer the improperlyformatted question.

As another specific illustrative example, in one embodiment, an askinguser's question data is analyzed as it is being entered into thequestion and answer based customer support system. If the question isdetermined to be an improperly formatted question because the questionis a general knowledge/open-ended type/format question, then the askinguser is provided format transformation instructions that guide the userthrough a step-by-step process to transform the identified generalknowledge/open-ended format question into the most highly rated generalknowledge/open-ended format in order of effectiveness (see FIG. 1A),i.e.: “Where” type/format questions, “What” type/format questions,“When” type/format questions, “Who” type/format questions, and “How”type/format questions, in that order.

As another specific illustrative example, in one embodiment, an askinguser's question is analyzed as it is being entered into the question andanswer based customer support system and if the question is determinedto be an improperly formatted question because the question is in a lowquality rhetorical, or an otherwise “unanswerable”, question format, theasking user is provided format transformation instructions to guide theuser through a step-by-step process to transform the identifiedrhetorical, or unanswerable, improperly formatted question into aproperly formatted question in a high quality question format, such as,for example, a closed-ended question, capable of being answered with asimple “yes” or “no”, or a closed-ended question capable of beinganswered by multi-choice, or mapping. In one embodiment, the formattransformation instructions are used to implement this the step-by-steptransformation process before the question is submitted to the questionand answer based customer support system for response, and before anyresources are devoted to actually trying to answer the improperlyformatted question.

As another specific illustrative example, in one embodiment, an askinguser's question, and/or a search query submitted by a searching user, isanalyzed as it is being entered into the question and answer basedcustomer support system to identify improperly formatted questions withlow quality question formats indicating grammatically incorrectquestions and/or queries. In one embodiment, if a question and/or searchquery is determined to an improperly formatted question because thequestion or query is determined to be in a low quality grammaticallyincorrect question or search query format, the asking or searching useris provided format transformation instructions and thereby guidedthrough a step-by-step process to transform the improperly formattedquestion or search query into a properly formatted question having ahigh quality grammatically correct format. In one embodiment, the formattransformation instructions are used to implement this the step-by-steptransformation process before the question is submitted to the questionand answer based customer support system for response, and before anyresources are devoted to actually trying to answer the improperlyformatted question.

Using the concepts disclosed herein, satisfaction with the answersprovided through a question and answer based customer support system arepredicted before the question is formally submitted to the question andanswer based customer support system. Therefore the concepts disclosedherein provide an opportunity to intervene in the question draftingprocess, in relative real time, while the question is still beingformulated, and before any resources are devoted to trying to answer lowquality questions. Consequently, in one embodiment, customized formattransformation instructions are provided so that the user is coachedduring the user's question formulation, i.e., during the user's entry ofthe data representing the question. In this way there is a significantlyhigher likelihood that not only the asking user will be satisfied withthe answer eventually provided, but that other searching users accessingthe question and answer pair content through a question and answerdatabase at a later time will also be satisfied with the answer content.Consequently, using the method and system for pro-active detection andcorrection of low quality questions submitted to a question and answerbased customer support system discussed herein, a method and system isprovided for reliably and efficiently predicting answer quality, anduser satisfaction with a potential answer to the user's question beforethe question is submitted to the support community for response, andbefore any resources are devoted to trying to answer the question. Inthis way, not only can the individual asking user's satisfaction with ananswer be predicted before the answer content is provided to the askinguser, but the satisfaction of other searching users with the questionand answer content can be predicted to ensure answers likely to resultin poor user satisfaction ratings are never provided to the searchingusers.

Therefore the disclosed method and system for pro-active detection andcorrection of low quality questions submitted to a question and answerbased customer support system provides for significant improvements tothe technical fields of customer support, information dissemination,software implementation, and user experience. In addition, using thedisclosed method and system for pro-active detection and correction oflow quality questions submitted to a question and answer based customersupport system results in more efficient use of human and non-humanresources, fewer processor cycles being utilized, reduced memoryutilization, and less communications bandwidth being utilized to relaydata to and from backend systems. As a result, computing systems aretransformed into faster, more efficient, and more effective computingsystems by implementing the method and system for pro-active detectionand correction of low quality questions submitted to a question andanswer based customer support system disclosed herein.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A shows a table of results data obtained through analysis ofactual questions submitted to a question and answer based customersupport system indicating question types, the frequency of the questiontypes as a percentage of questions asked, and ranking of the questiontypes by up vote fraction;

FIG. 1B is a graph of results data obtained through analysis of actualquestions submitted to a question and answer based customer supportsystem showing the discovered relationship between “subject matterquestions,” “product related questions,” and the category of questionasked;

FIG. 1C is a table of results data obtained through analysis of actualquestions submitted to a question and answer based customer supportsystem showing the Wald Chi-square statistics for the top subjectattributes of a user vote prediction model;

FIG. 2A is an illustrative example of a first question transformationinterface screen used to provide users format transformationinstructions that direct users in transforming improperly formattedquestions into properly formatted closed-ended questions in accordancewith one embodiment;

FIG. 2B is an illustrative example of a second question transformationinterface screen used to provide users format transformationinstructions and using a question optimizer approach to direct userstowards transforming improperly formatted questions into properlyformatted questions in accordance with one embodiment;

FIG. 2C is an illustrative example of a third question transformationinterface screen used to provide users format transformationinstructions that direct users towards transforming improperly formattedquestions into properly formatted questions and including a visualquestion-quality meter, which provides a simple read of question-qualityin accordance with one embodiment;

FIG. 3 is a flow chart representing one example of a generalized processfor pro-active detection and correction of low quality questionssubmitted to a question and answer based customer support system inaccordance with one embodiment;

FIGS. 4A, 4B, and 4C together are a block diagram depicting a questionanalysis and reformation process for using format transformationinstructions to transform an improperly formatted question into aproperly formatted question in accordance with one embodiment; and

FIG. 5 is a block diagram of a hardware and production environmentsystem 500 for providing a process for pro-active detection andcorrection of low quality questions submitted to a question and answerbased customer support system in accordance with one embodiment.

Common reference numerals are used throughout the FIG.s and the detaileddescription to indicate like elements. One skilled in the art willreadily recognize that the above FIG.s are examples and that otherarchitectures, modes of operation, orders of operation, andelements/functions can be provided and implemented without departingfrom the characteristics and features of the invention, as set forth inthe claims.

TERM DEFINITIONS

Herein, a software system can be, but is not limited to, any datamanagement system implemented on a computing system, accessed throughone or more servers, accessed through a network, accessed through acloud, and/or provided through any system or by any means, as discussedherein, and/or as known in the art at the time of filing, and/or asdeveloped after the time of filing, that gathers/obtains data, from oneor more sources and/or has the capability to analyze at least part ofthe data.

As used herein, the term software system includes, but is not limited tothe following: computing system implemented, and/or online, and/orweb-based, personal and/or business tax preparation systems; computingsystem implemented, and/or online, and/or web-based, personal and/orbusiness financial management systems, services, packages, programs,modules, or applications; computing system implemented, and/or online,and/or web-based, personal and/or business management systems, services,packages, programs, modules, or applications; computing systemimplemented, and/or online, and/or web-based, personal and/or businessaccounting and/or invoicing systems, services, packages, programs,modules, or applications; and various other personal and/or businesselectronic data management systems, services, packages, programs,modules, or applications, whether known at the time of filling or asdeveloped later.

Specific examples of software systems include, but are not limited tothe following: TurboTax™ available from Intuit, Inc. of Mountain View,Calif.; TurboTax Online™ available from Intuit, Inc. of Mountain View,Calif.; Quicken™, available from Intuit, Inc. of Mountain View, Calif.;Quicken Online™, available from Intuit, Inc. of Mountain View, Calif.;QuickBooks™, available from Intuit, Inc. of Mountain View, Calif.;QuickBooks Online™, available from Intuit, Inc. of Mountain View,Calif.; Mint™, available from Intuit, Inc. of Mountain View, Calif.;Mint Online™, available from Intuit, Inc. of Mountain View, Calif.;and/or various other software systems discussed herein, and/or known tothose of skill in the art at the time of filing, and/or as developedafter the time of filing.

As used herein, the terms “computing system,” “computing device,” and“computing entity,” include, but are not limited to, the following: aserver computing system; a workstation; a desktop computing system; amobile computing system, including, but not limited to, smart phones,portable devices, and/or devices worn or carried by a user; a databasesystem or storage cluster; a virtual asset; a switching system; arouter; any hardware system; any communications system; any form ofproxy system; a gateway system; a firewall system; a load balancingsystem; or any device, subsystem, or mechanism that includes componentsthat can execute all, or part, of any one of the processes and/oroperations as described herein.

In addition, as used herein, the terms “computing system” and “computingentity,” can denote, but are not limited to the following: systems madeup of multiple virtual assets, server computing systems, workstations,desktop computing systems, mobile computing systems, database systems orstorage clusters, switching systems, routers, hardware systems,communications systems, proxy systems, gateway systems, firewallsystems, load balancing systems, or any devices that can be used toperform the processes and/or operations as described herein.

Herein, the terms “mobile computing system” and “mobile device” are usedinterchangeably and include, but are not limited to the following: asmart phone; a cellular phone; a digital wireless telephone; a tabletcomputing system; a notebook computing system; any portable computingsystem; a two-way pager; a Personal Digital Assistant (PDA); a mediaplayer; an Internet appliance; devices worn or carried by a user; or anyother movable/mobile device and/or computing system that includescomponents that can execute all, or part, of any one of the processesand/or operations as described herein.

Herein, the term “production environment” includes the variouscomponents, or assets, used to deploy, implement, access, and use, agiven software system as that software system is intended to be used. Invarious embodiments, production environments include multiple computingsystems and/or assets that are combined, communicatively coupled,virtually and/or physically connected, and/or associated with oneanother, to provide the production environment implementing theapplication.

As specific illustrative examples, the assets making up a givenproduction environment can include, but are not limited to, thefollowing: one or more computing environments used to implement at leastpart of the software system in the production environment such as a datacenter, a cloud computing environment, a dedicated hosting environment,and/or one or more other computing environments in which one or moreassets used by the application in the production environment areimplemented; one or more computing systems or computing entities used toimplement at least part of the software system in the productionenvironment; one or more virtual assets used to implement at least partof the software system in the production environment; one or moresupervisory or control systems, such as hypervisors, or other monitoringand management systems used to monitor and control assets and/orcomponents of the production environment; one or more communicationschannels for sending and receiving data used to implement at least partof the software system in the production environment; one or more accesscontrol systems for limiting access to various components of theproduction environment, such as firewalls and gateways; one or moretraffic and/or routing systems used to direct, control, and/or bufferdata traffic to components of the production environment, such asrouters and switches; one or more communications endpoint proxy systemsused to buffer, process, and/or direct data traffic, such as loadbalancers or buffers; one or more secure communication protocols and/orendpoints used to encrypt/decrypt data, such as Secure Sockets Layer(SSL) protocols, used to implement at least part of the software systemin the production environment; one or more databases used to store datain the production environment; one or more internal or external servicesused to implement at least part of the software system in the productionenvironment; one or more backend systems, such as backend servers orother hardware used to process data and implement at least part of thesoftware system in the production environment; one or more softwaremodules/functions used to implement at least part of the software systemin the production environment; and/or any other assets/components makingup an actual production environment in which at least part of thesoftware system is deployed, implemented, accessed, and run, e.g.,operated, as discussed herein, and/or as known in the art at the time offiling, and/or as developed after the time of filing.

As used herein, the term “computing environment” includes, but is notlimited to, a logical or physical grouping of connected or networkedcomputing systems and/or virtual assets using the same infrastructureand systems such as, but not limited to, hardware systems, softwaresystems, and networking/communications systems. Typically, computingenvironments are either known, “trusted” environments or unknown,“untrusted” environments. Typically, trusted computing environments arethose where the assets, infrastructure, communication and networkingsystems, and security systems associated with the computing systemsand/or virtual assets making up the trusted computing environment, areeither under the control of, or known to, a party.

In various embodiments, each computing environment includes allocatedassets and virtual assets associated with, and controlled or used tocreate, and/or deploy, and/or operate at least part of the softwaresystem.

In various embodiments, one or more cloud computing environments areused to create, and/or deploy, and/or operate at least part of thesoftware system that can be any form of cloud computing environment,such as, but not limited to, a public cloud; a private cloud; a virtualprivate network (VPN); a subnet; a Virtual Private Cloud (VPC); asub-net or any security/communications grouping; or any othercloud-based infrastructure, sub-structure, or architecture, as discussedherein, and/or as known in the art at the time of filing, and/or asdeveloped after the time of filing.

In many cases, a given software system or service may utilize, andinterface with, multiple cloud computing environments, such as multipleVPCs, in the course of being created, and/or deployed, and/or operated.

As used herein, the term “virtual asset” includes any virtualized entityor resource, and/or virtualized part of an actual, or “bare metal”entity. In various embodiments, the virtual assets can be, but are notlimited to, the following: virtual machines, virtual servers, andinstances implemented in a cloud computing environment; databasesassociated with a cloud computing environment, and/or implemented in acloud computing environment; services associated with, and/or deliveredthrough, a cloud computing environment; communications systems usedwith, part of, or provided through a cloud computing environment; and/orany other virtualized assets and/or sub-systems of “bare metal” physicaldevices such as mobile devices, remote sensors, laptops, desktops,point-of-sale devices, etc., located within a data center, within acloud computing environment, and/or any other physical or logicallocation, as discussed herein, and/or as known/available in the art atthe time of filing, and/or as developed/made available after the time offiling.

In various embodiments, any, or all, of the assets making up a givenproduction environment discussed herein, and/or as known in the art atthe time of filing, and/or as developed after the time of filing can beimplemented as one or more virtual assets.

In one embodiment, two or more assets, such as computing systems and/orvirtual assets, and/or two or more computing environments are connectedby one or more communications channels including but not limited to,Secure Sockets Layer (SSL) communications channels and various othersecure communications channels, and/or distributed computing systemnetworks, such as, but not limited to the following: a public cloud; aprivate cloud; a virtual private network (VPN); a subnet; any generalnetwork, communications network, or general network/communicationsnetwork system; a combination of different network types; a publicnetwork; a private network; a satellite network; a cable network; or anyother network capable of allowing communication between two or moreassets, computing systems, and/or virtual assets, as discussed herein,and/or available or known at the time of filing, and/or as developedafter the time of filing.

As used herein, the term “network” includes, but is not limited to, anynetwork or network system such as, but not limited to, the following: apeer-to-peer network; a hybrid peer-to-peer network; a Local AreaNetwork (LAN); a Wide Area Network (WAN); a public network, such as theInternet; a private network; a cellular network; any general network,communications network, or general network/communications networksystem; a wireless network; a wired network; a wireless and wiredcombination network; a satellite network; a cable network; anycombination of different network types; or any other system capable ofallowing communication between two or more assets, virtual assets,and/or computing systems, whether available or known at the time offiling or as later developed.

As used herein, the term “user experience” includes not only the dataentry and question submission process, but also other user experiencefeatures provided or displayed to the user such as, but not limited tothe following: interfaces; images; backgrounds; avatars; highlightingmechanisms; icons; and any other features that individually, or incombination, create a user experience, as discussed herein, and/or asknown in the art at the time of filing, and/or as developed after thetime of filing.

Herein, the term “party,” “user,” “user consumer,” and “customer” areused interchangeably to denote any party and/or entity that interfaceswith, and/or to whom information is provided by, the method and systemfor pro-active detection and correction of low quality questionssubmitted to a question and answer based customer support systemdescribed herein, and/or a person and/or entity that interfaces with,and/or to whom information is provided by, the method and system forpro-active detection and correction of low quality questions submittedto a question and answer based customer support system described herein,and/or a legal guardian of person and/or entity that interfaces with,and/or to whom information is provided by, the method and system forpro-active detection and correction of low quality questions submittedto a question and answer based customer support system described herein,and/or an authorized agent of any party and/or person and/or entity thatinterfaces with, and/or to whom information is provided by, the methodand system for pro-active detection and correction of low qualityquestions submitted to a question and answer based customer supportsystem described herein. For instance, in various embodiments, a usercan be, but is not limited to, a person, a commercial entity, anapplication, a service, and/or a computing system.

As used herein, the term “asking user” includes a user of a softwaresystem submitting a question to a question and answer based customersupport system and the term “searching user” includes a user of asoftware system submitting a search query to a customer support questionand answer database associated with a question and answer based customersupport system.

Theory and Empirical Analysis

The embodiments disclosed herein were developed to incorporate theoriesand address relationships discovered through analysis of data collectedfrom a specific question and answer based customer support systemimplemented by Intuit™ Inc. of Mountain View, Calif. The specificquestion and answer based customer support system through which the datawas collected was the TurboTax™ AnswerXchange™ (AXC) question and answerbased customer support system.

AXC is a social question and answer based customer support systemproviding support for TurboTax™ customers and also serving as adiscussion forum in the area of US Federal and State taxation. AXC isalso used to generate reusable content for TurboTax™ user searches,i.e., to create a customer support question and answer database forTurboTax™ users. In fact, only 1.5% of AXC users are “asking users” whoactually submit questions, while the remaining “searching users” lookfor answers by searching a customer support question and answer databaseprovided through AXC.

AXC includes a support community of customer support personnel. In oneembodiment, questions submitted to AXC are answered by members of thesupport community of customer support personnel. In one embodiment, thecustomer support personnel include paid professional support personnelin the employ of Intuit™ and volunteer, often non-paid, expert users ofthe TurboTax™ software system. In one embodiment, the volunteer expertusers of the TurboTax™ software system are identified and certified byIntuit™.

Questions submitted to AXC were formulated in a variety of ways anddirected to various broad categories. As one example, some questionswere “product related questions”, e.g., questions related to pricing,installation, version choice, etc. of the TurboTax™ software system thatoften had little or no relation to the subject matter/endeavor supportedby the TurboTax™ software system, i.e., tax preparation. On the otherhand, some questions were “subject matter related,” or substantivequestions, directly related to the subject matter/endeavor supported bythe TurboTax™ software system, i.e., Federal and State taxation and taxpreparation.

As an example, the questions “What version of TurboTax™ should I use?”or “How do I install TurboTax™?” would be product related questionswhile the questions “Can I deduct my computer?” or “What is my adjustedgross income?” would be subject matter related questions. As discussedbelow, it was empirically determined that, in general, product relatedquestions are best answered by paid support personnel in the employ ofIntuit™ while subject matter related questions are often best answeredby volunteer expert users.

Similar to other question and answer based customer support systems, AXCmeasures the quality of content, and answer content in particular, bycollecting statistics of up and down votes directed to answer contentprovided by the asking users and searching users where an up voteindicates user satisfaction with the answer to the question and a downvote indicates user dissatisfaction with the answer to the question.

At the same time, the AXC questions were not ranked or judged based onquality of content beyond user satisfaction ratings, unless the questionwas determined as inappropriate and blocked from AXC. Therefore, usersatisfaction with answer content in AXC typically would be derived fromuser votes alone thus providing a useful metric for answer quality. Forexample, this approach was applied to predicting answer satisfaction inAXC based on the one or more attributes of the question and answercombined with one or more AXC users' attributes. On the other hand, asdisclosed herein, a recent analysis of AXC vote statistics found thatanswer quality/satisfaction is largely predetermined by the questionsubject matter, and/or type/format, and that users' satisfaction votescan be predicted with reasonable accuracy based on the attributes of thequestion alone. This finding provided a practical framework for“pro-active” detection of low-quality content at the question submissionstage, i.e. before the question is even answered, and is the foundationof the disclosed embodiments of methods and systems for pro-activedetection and correction of low quality questions submitted to aquestion and answer based customer support system.

As noted above, questions submitted to AXC were formulated in a varietyof ways and directed to various broad categories and subjects and invarious question formats representing various question types. As oneexample, one user may ask “What version of TurboTax should I use?”Another user may ask “How do I install TurboTax?” Yet another user mayask “Can I deduct my computer?” It was determined that the optimal AXCdesign must be based on an empiric question taxonomy taking into accountone or more of, the question category, the question format, the questiongrammar structure, the type of anticipated answer, asker type, andvarious other factors.

We began with general knowledge/open-ended question taxonomy.Specifically, we looked for “Wh- word” and “How” questions includinginterrogative pronouns such as “Who”, “What”, “When”, “Where”, “Why” and“How” used to compose general knowledge/open-ended category questions.

FIG. 1A is a table of results data obtained through the analysis ofactual questions submitted to AXC. The table of FIG. 1A shows questiontypes, also referred to herein as formats (column 101) of the 2014 AXCquestions received, the frequency of the question types as a percentageof questions asked (column 102), and ranking of the questiontypes/formats by up vote fraction (column 103) that are shown in FIG. 1Ain the descending order. The sum of up vote and down vote fractions isequal to 100%. As seen in FIG. 1A, when “Wh- words” analysis was appliedto the AXC question subjects with question summaries limited to 255characters, 54.5% of the questions received fell into the generalknowledge/open-ended category.

One of our goals was to separate question types/formats by the observedstatistics relating up and down votes associated with the answersprovided to each question types/formats. The result of this analysis isshown in FIG. 1A. Referring to FIG. 1A, note that “Why” formattedquestions can often indicate mildly negative sentiment and often imply arhetorical question, e.g., “Why is this so difficult?” or “Why is thistaking so much time?” The inventors postulate that this explains thelowest up vote fraction of 56.3% being associated with the “Why”question type/format in the general knowledge/open-ended category, asshown in FIG. 1A.

Next, we selected closed-ended category questions from the 56.8% of AXCquestions that did not belong to the general knowledge/open-endedcategory. Most of the closed-ended type/format questions in AXC were inthe sub-category of “Yes-No” type/format questions. These “Yes-No”type/format questions typically start with an auxiliary verb such as“Do”, “Can”, “Be.” As indicated by the name, the “Yes-No” type/formatquestions can be answered by a “Yes” or “No” answer. A specificillustrative example of a “Yes-No” questions would be: “Can I deduct mycomputer?” with the possible answers “Yes, you can” or “No you can't.”

The second sub-category of closed-ended question type/format includes“Choice” type/format questions. “Choice” type/format questions generallystart with an auxiliary verb as well, but also contain the conjunction“or.” Consequently, “Choice” type/format questions usually result in amultiple choice answer embedded in the body of the question. A specificillustrative example of a “Choice” type/format question would be “ShouldI install X or Y version?” with the possible answers “You need toinstall Y,” “You need to install X,” “You need to install either X orY,” “You need to install neither X or Y.”

As seen in FIG. 1A, closed-ended type/format questions resulted in animpressive 85.9% up vote, i.e., 85.9% of users who submitted questionsin the closed-ended format were satisfied with the answer provided. Asseen in FIG. 1A, this was the highest satisfaction rating of allquestion types/formats. The high fraction of the up votes for theanswers to the closed-ended type/format questions of in FIG. 1A is notsurprising. Closed-ended type/format questions were typically longenough to provide sufficient context for answering, and were more likelyto be subject matter related questions, as opposed to product relatedquestions. As discussed below, subject matter related questions werechanneled to volunteer expert users for answering and had a higherpredicted likelihood of resulting in an up vote (see FIG. 1B discussedbelow).

Finally, if a question submitted to AXC was deemed to be neither ageneral knowledge/open-ended nor a closed-ended type/format question,the question was classified as being in the ill-formed question categoryby default. Most of the ill-formed category questions did not follow agrammatically correct question format either intentionally, e.g., searchquery type, or unintentionally, e.g., improper grammar, punctuation,etc., and were more difficult to answer. This, of course resulted in ahigher probability of down vote from the users.

“How” and “Why” question types/formats were detectable with regularexpressions analysis. Similarly “When”, “Where” and “who” questiontypes/formats were detectable with regular expressions analysis but theanalysis was slightly more involved as it typically requiredpart-of-speech tagging to avoid confusion with relative adverbs andrelative pronouns. However, as seen in FIG. 1A, these questiontypes/formats were less common in AXC. More exotic questiontypes/formats, such as “tag questions,” “leading questions,” and“embedded questions,” were determined to be extremely rare in AXC andtherefore were largely excluded from our analysis.

FIG. 1B is a graph of results data obtained through analysis of actualquestions submitted to AXC and showing the discovered relationshipbetween: the percentage of up votes indicating user satisfaction withthe answer provided (vertical axis), the category of question, e.g.,“subject matter questions”, or in the case of AXC, tax questions;“product related questions,” or in the case of AXC TurboTax™ productquestions; and the type/format of the question asked. The horizontalaxis in FIG. 1B was divided into Deciles 1 to 10, with Decile 1corresponding to well-defined subject matter related questions, andDecile 10 corresponding to well-defined product related questions.Consequently, FIG. 1B displays the satisfaction percentages of generalknowledge/open-ended (K), closed-ended (C), and ill-formed questionstypes versus content type. In our analysis, only voted upon, andtherefore answered, questions were used to ensure the resultsreported/displayed in FIG. 1B consistent with the resultsreported/displayed in FIG. 1A.

FIG. 1C is a table of results data obtained through analysis of actualquestions submitted to AXC showing the Wald Chi-square statistics forthe top subject attributes of an AXC user vote prediction model. In FIG.1C, the (+) and (−) signs indicate positive or negative correlationbetween attribute and up vote. As seen in FIG. 1C, closed-ended, “Why”and “How” question types are three out of the four most important modelattributes. The third attribute, “Reporting a problem,” was found tocorrelate with “Why” and “How” types. As noted above, “Why” questionsare often rhetorical and may remain “unanswerable” unless the userprovides further details.

Once the data of FIG. 1A, FIG. 1B, and FIG. 1C was obtained, an initialgoal of correlating the different question types/formats with theobserved statistics relating up and down votes associated with theanswers provided to each question type/format was attained. Then threeadditional goals were identified: transforming “Why” and “How”type/format questions into to closed-ended type/format questions;transforming “unanswerable” questions into “answerable” questions; andtransforming ill-formed questions into “well formed” questions.

With respect to the goal of transforming “Why” and “How” type/formatquestions into to closed-ended type/format questions, it was determinedthat the biggest positive and negative impacts on AXC user satisfactioncame from the answers to closed-ended and “how/why” type/formatquestions, respectively. While it is difficult to alter the broadcategory/subject of the question, e.g., switching user attention fromproduct related questions to subject matter related questions, it wasdetermined that it may be possible to transform the question type/formatfrom a low quality format question, with a low predicted usersatisfaction with any answer to the question, to a high quality formatquestion, with a higher predicted user satisfaction with any answer tothe question. For example, asking the user to re-phrase/transform a“Why” type/format question into a closed-ended type/format question.

With respect to the goal of transforming “unanswerable” questions into“answerable” questions, it was determined that the unanswerablequestions were often formed as a statement without specific details. Thetask therefore involved a re-phrasing/transformation process similar tothe process for transforming “Why” and “How” type/format questions intoto closed-ended type/format questions, and also asking the user for moreinformation.

With respect to the goal of transforming ill-formed questions into “wellformed” questions, it was determined that re-phrasing/transformingill-formed questions, e.g. making the questions more detailed and addingproper punctuation, may result in better answers. See FIG. 1B.

To address these three goals, we designed and tested threequestion-asking experience question transformation interface screensused to provide format transformation instructions that direct userstowards transforming improperly formatted questions into properlyformatted closed-ended questions. The three question-asking experiencequestion transformation interface screens are shown in FIGS. 2A, 2B, and2C and discussed separately below.

The first experience prototype, and associated question transformationinterface screen, we explored is shown in FIG. 2A. The experienceprototype, and associated question transformation interface screen, ofFIG. 2A used constraints to direct users towards asking closed-endedquestions, and went as far as defaulting to Yes/No answer types/formats.The experience prototype, and associated question transformationinterface screen, of FIG. 2A emphasized closed-ended questiontypes/formats, which yield the highest rated answers (see FIG. 1A). Thisapproach was ultimately abandoned because of the front-loaded cognitiveoverhead it created that forced users to think about their questiontype/format before they had a chance to even phrase it, and whichultimately proved too burdensome on the user.

It was found survey-style answer choices helped phrasing. For open-endedgeneral knowledge/open-ended questions, we prompted with the most highlyrated question-types/formats in order of effectiveness (see FIG. 1A),i.e.: “Where” type/format questions, “What” type/format questions,“When” type/format questions, “Who” type/format questions, and “How”type/format questions. We left out “Why” type/format questions since, asdiscussed above, “Why” type/format questions often lead to“unanswerable” or rhetorical questions.

The second experience prototype, and associated question transformationinterface screen, we explored is shown in FIG. 2B and is referred to asthe question optimizer approach. The question optimizer experienceprototype, and associated question transformation interface screen, ofFIG. 2B allows the user to formulate/phrase a question first, then theuser is provided the format transformation instructions advising theuser on how to re-phrase/transform an improperly formatted question intoa properly formatted question. The question optimizer experienceprototype, and associated question transformation interface screen, ofFIG. 2B thus provides contextual cues to the user to help the userre-phrase the question in such a way as to achieve the highest predictedlevel of satisfaction with any answer to that question using the data ofFIG. 1A. Using the one embodiment of the question optimizer experienceprototype, and associated question transformation interface screen, ofFIG. 2B, users are asked to retype their question rather than edit thequestion. Research confirmed that this approach helps the usersre-phrase the question more effectively.

One advantage of the question optimizer experience prototype, andassociated question transformation interface screen, approach of FIG. 2Bis that initial question data inputs from the user can be analyzedpro-actively in real time. In most cases, the question type/format couldbe reliably detected from the first few characters/tokens/text of thequestion entered, as the question was being entered. The interventionaccomplished through the question optimizer experience prototype, andassociated question transformation interface screen, of FIG. 2B maytherefore come at the very early stages of question formulation,alleviating the user's concern about accidentally losing the questionbefore it is submitted to/recorded in the AXC customer support questionand answer database.

To test the question optimizer experience prototype, and associatedquestion transformation interface screen, approach of FIG. 2B, we used40 AXC “Why” type/format questions belonging to the bottom 10% questionquality. The study participants were Intuit™ employees and Peet's Coffee& Tea™ customers who were shown the “Why” formatted questions inside thequestion optimizer experience prototype, and associated questiontransformation interface screen, of FIG. 2B. Samples of the original andre-phrased questions are shown in the following Examples 1, 2 and 3.

Example 1 Transformation from “why” Type/Format Question to “What”Type/Format Question

Original Question:

I don't understand why I can't efile”

Re-Phrased Question:

“What steps do I need to take to efile?”

Example 2 Transformation from “why” Type/Format Question to Closed-EndedType/Format Question

Original Question:

“why is my debt card being declined”

Re-Phrased Question:

“My Debit card has been declined. Is there something I need to do tomake it work?”

Example 3 Transformation from “why” Type/Format Question to “how”Type/Format Question

Original Question:

“why does the program freeze up when I try to download a state?”

Re-Phrased Question:

“When I try to download a stat the program is freezing. How can I fixit?”

The overall success rate of the question optimizer experience prototype,and associated question transformation interface screen, of FIG. 2B,i.e., the ability of the user to transform a “Why” type/format questionto another, preferred, question type/format was 80%. What was especiallyencouraging is that some users were able to re-phrase “Why” type/formatquestions into closed-ended category questions (Example 2) while keepingthe original intent of the question intact. This allowed us toaccomplish stated Goal 1, i.e., transforming “Why” and “How”type/formatted questions into to closed-ended category/formattedquestions.

In addition, in most cases, the questions transformed/re-phrased usingthe question optimizer experience prototype, and associated questiontransformation interface screen, of FIG. 2B, became easier to understandand “answerable.” This allowed us to accomplish stated Goal 2, i.e.,transforming “unanswerable” questions into “answerable” questions.

In addition, re-phrasing/transforming using the question optimizerexperience prototype, and associated question transformation interfacescreen, of FIG. 2B, typically resulted in better formed/formattedquestions compared to the original question, including proper spelling,grammar structure, capitalization and punctuation. This allowed us toaccomplish stated Goal 3, i.e., transforming ill-formed questions into“well formed” questions.

In another experiment, we also found that users who were asked tore-type the question using the question optimizer experience prototype,and associated question transformation interface screen, of FIG. 2B,generally did a better job in re-phrasing the original question. On thecontrary, users who were asked to edit the original question typicallykept the original question type intact.

The third experience prototype, and associated question transformationinterface screen, we explored is shown in FIG. 2C which abandons theneed to choose between general knowledge/open-ended or closed-endedtype/format questions upfront. This allows the user to submit/documenttheir question, and then with that task complete, move on tooptimizing/transforming it. The experience prototype, and associatedquestion transformation interface screen, of FIG. 2C also introduces theconcept of a visual question-quality meter 201, which provides a simplerread of question-quality. We believe users will be more interestedin-rephrasing/transforming their question multiple times in a quest toget the meter “into the green.”

The benefits of the data driven AXC question-asking experience, and theresulting method and system for pro-active detection and correction oflow quality questions submitted to a question and answer based customersupport system disclosed herein, are multifold. Better-formulatedquestions improve overall user experience and increase chances ofreceiving better answers contributing to the user satisfaction. Moreimportantly, new questions will be both more relevant and higher qualityfor the 98.5% of customers who are searching users that leverageexisting answers.

The data analysis discussed above provides a compelling argument for theidea that re-phrasing general knowledge/open-ended questions toclosed-ended questions using the method and system for pro-activedetection and correction of low quality questions submitted to aquestion and answer based customer support system disclosed herein, willresult in higher quality question and answer based customer supportsystem content, thus providing higher satisfaction for both the askingand searching user.

DETAILED DISCLOSURE

Embodiments will now be discussed with reference to the accompanyingFIG.s, which depict one or more exemplary embodiments. Embodiments maybe implemented in many different forms and should not be construed aslimited to the embodiments set forth herein, shown in the FIG.s, and/ordescribed below. Rather, these exemplary embodiments are provided toallow a complete disclosure that conveys the principles of theinvention, as set forth in the claims, to those of skill in the art.

In one embodiment a software system is provided. As noted above, herein,the term software system includes, but is not limited to the following:computing system implemented, and/or online, and/or web-based, personaland/or business tax preparation systems; computing system implemented,and/or online, and/or web-based, personal and/or business financialmanagement systems, services, packages, programs, modules, orapplications; computing system implemented, and/or online, and/orweb-based, personal and/or business management systems, services,packages, programs, modules, or applications; computing systemimplemented, and/or online, and/or web-based, personal and/or businessaccounting and/or invoicing systems, services, packages, programs,modules, or applications; and various other personal and/or businesselectronic data management systems, services, packages, programs,modules, or applications, whether known at the time of filling or asdeveloped later.

Specific examples of software systems include, but are not limited tothe following: TurboTax™ available from Intuit, Inc. of Mountain View,Calif.; TurboTax Online™ available from Intuit, Inc. of Mountain View,Calif.; Quicken™, available from Intuit, Inc. of Mountain View, Calif.;Quicken Online™, available from Intuit, Inc. of Mountain View, Calif.;QuickBooks™, available from Intuit, Inc. of Mountain View, Calif.;QuickBooks Online™, available from Intuit, Inc. of Mountain View,Calif.; Mint™, available from Intuit, Inc. of Mountain View, Calif.;Mint Online™, available from Intuit, Inc. of Mountain View, Calif.;and/or various other software systems discussed herein, and/or known tothose of skill in the art at the time of filing, and/or as developedafter the time of filing.

In one embodiment a question and answer based customer support system,e.g., a social question-and-answer (Q&A) system, is provided to supportusers of the software system.

In one embodiment, the question and answer based customer support systemserves as a discussion forum in an area of endeavor related to thesoftware system. As an illustrative example, in one embodiment, thequestion and answer based customer support system is provided to supporta tax preparation software system and therefore the discussion forum isrelated to “federal and state taxation and tax preparation.”

In one embodiment, users of the software system are provided thecapability to submit questions regarding the installation,implementation, use and operation of the software system through thequestion and answer based customer support system.

In one embodiment, the question and answer based customer support systemincludes a support community of customer support personnel. In oneembodiment, the customer support personnel include paid supportpersonnel in the employ of the provider of the software system andnon-paid volunteer expert users of the software system. In oneembodiment, the volunteer expert users of the software system areidentified and certified by the provider of the software system.

In one embodiment, through the question and answer based customersupport system, users of the software system are provided the capabilityto submit questions to members of the support community of customersupport personnel.

In one embodiment, questions submitted to the question and answer basedcustomer support system can be related to very different broadcategories, be of various question types, and be formatted in variousdifferent ways. For example, some questions submitted to the questionand answer based customer support system are product related questions,e.g., questions related to pricing, installation, version choice, etc.for the software systems that often have little or no relation to thesubject matter/job of the software system, i.e., the endeavor supportedby the software system. On the other hand, some questions submitted tothe question and answer based customer support system are subject matterrelated, or substantive, questions directly related to the subjectmatter/endeavor of the software system.

As an illustrative example, in the case of a tax preparation softwaresystem, the questions “What version of the tax preparation softwaresystem should I use?” or “How do I install the tax preparation softwaresystem?” would be product related questions while the questions “Can Ideduct my computer?” or “What is my adjusted gross income?” would besubject matter related questions.

In general, product related questions are best answered by paid supportpersonnel in the employ of the provider of the software system whilesubject matter related questions are often best answered by volunteerexpert users of the software system. Consequently, in one embodiment, itis desirable to identify the broad category/subject matter of thequestions, e.g., product related questions and subject matter relatedquestions, and route the questions accordingly either to supportpersonnel or volunteer expert users of the software system.

In one embodiment, the question and answer based customer support systemis used to generate reusable content for the question and answer basedcustomer support system, e.g., the question and answer based customersupport system is used to generate a customer support question andanswer database. In one embodiment, the creation of the customer supportquestion and answer database is the primary goal of the question andanswer based customer support system. This is because it has beenempirically demonstrated that only 1.5% of users of a typical questionand answer based customer support system are “asking users” who submittheir own questions, while the remaining 98.5% of users are “searchingusers” who look for answers by searching for similar topics andquestions answered in the customer support question and answer database.

As discussed below in more detail, questions submitted to the questionand answer based customer support system can also bestructured/formatted in a variety of ways and these various questiontype/formats can vary significantly in quality, and more importantly, inthe predicted user satisfaction with an answer, any answer, to thequestion.

As a specific illustrative example, questions submitted to the questionand answer based customer support system can be, but are not limited to:general knowledge/open-ended type questions, defined as “Who”type/format questions, “What” type/format questions, “When” type/formatquestions, “Where” type/format questions, “Why” type/format questions,and “How” type/format questions; rhetorical, or otherwise “unanswerable”questions; grammatically incorrect questions and/or queries; otherwiseill-formed questions; and/or closed-ended questions, capable of beinganswered with a simple “yes” or “no”, or via a multi-choice, or mapping.

As discussed below, each of these question structures is associated witha empirically calculated predictability that the answer to the question,whatever that answer may be, will be found satisfactory by the users,e.g., the asking user and/or searching users. As also discussed below,this discovery by the inventors is leveraged by the methods and systemsfor pro-active detection and correction of low quality questionssubmitted to a question and answer based customer support systemdisclosed herein to predict user satisfaction with answers that mayeventually be provided through the question and answer based customersupport system by performing pre-submission analysis of the type/format,and other attributes, of the question itself, rather than post questionsubmission analysis of an answer provided in response to the question.This paradigm shifting approach to predicting user satisfaction with ananswer based on the user's question alone, and before the answer isgenerated, is in direct contrast to prior art assumptions and approachesthat focused on the answers provided through a question and answer basedcustomer support system, and analysis performed after both the questionand answer had already been formulated and provided to users.

In one embodiment, low quality question formats that are predicted toresult in answers that will have user satisfaction ratings below athreshold level of user satisfaction are defined/identified.

As noted above, questions submitted to the question and answer basedcustomer support system can be formulated in a variety of ways, anddirected to various broad categories/subjects, such as “product relatedquestions” and “subject matter related,” or substantive questions,directly related to the subject matter/endeavor supported by thesoftware system. As also noted above, questions submitted to questionand answer based customer support system can be submitted in a varietyquestion types/formats. Consequently, in one embodiment, the method andsystem, for pro-active detection and correction of low quality questionssubmitted to a question and answer based customer support system isbased on an empiric question taxonomy taking into account one or moreof, the question type/format, the question grammar structure, the typeof anticipated answer, asker type, and various other factors.

In one embodiment, closed-ended category questions areidentified/defined. Most closed-ended formatted questions are placed inthe sub-category of “Yes-No” type questions. These “Yes-No” typequestions typically start with an auxiliary verb such as “Do”, “Can”,“Be.” As indicated by the name, the “Yes-No” type questions can beanswered by a “Yes” or “No” answer. A specific illustrative example of a“Yes-No” questions would be: “Can I deduct my computer?” with thepossible answers “Yes, you can” or “No you can't.”

The second sub-category of closed-ended question format includes“Choice” type questions. “Choice” type questions generally start with anauxiliary verb as well, but also contain the conjunction “or.”Consequently, “Choice” type questions usually result in a multiplechoice answer embedded in the body of the question. A specificillustrative example of a “Choice” type question would be “Should Iinstall X or Y version?” with the possible answers “You need to installY,” “You need to install X,” “You need to install either X or Y,” “Youneed to install neither X or Y.”

As seen in FIG. 1A, closed-ended questions result in an impressive 85.9%up vote, i.e., 85.9% of users who submit questions in the closed-endedformat are satisfied with the answer provided. As also seen in FIG. 1Athis is the highest satisfaction rating of all question formats.

In one embodiment, if a question submitted to the question and answerbased customer support system is deemed to be neither a generalknowledge/open-ended nor a closed-ended format question, the question isclassified as being in the ill-formed question category by default. Mostof the ill-formed category questions do not follow a grammaticallycorrect question format either intentionally, as in the case of a searchquery type, or unintentionally, e.g., wrong grammar, punctuation, etc.

“How” and “Why” question formats are detectable with format analysisinstructions that implement regular expressions analysis. Similarly“When”, “Where” and “Who” question formats are detectable with formatanalysis instructions that implement regular expressions analysis,however, the analysis is slightly more involved as it typically requirespart-of-speech tagging to avoid confusion with relative adverbs andrelative pronouns. As seen in FIG. 1A these question types are lesscommon in question and answer based customer support systems. Moreexotic question types such as “tag questions,” “leading questions,” and“embedded questions” are extremely rare in question and answer basedcustomer support systems, and therefore are largely excluded fromanalysis.

In one embodiment, low quality question formats that are predicted toresult in answers that will have user satisfaction ratings below athreshold level of user satisfaction are defined/identified based on theempirical data obtained as discussed above and shown in FIG. 1A.

In various embodiments, the threshold level of user satisfactionrequired to avoid being identified as a low quality question format canbe any threshold level as defined by the provider of the method andsystem for pro-active detection and correction of low quality questionssubmitted to a question and answer based customer support system, and/orthe provider of the question and answer based customer support system.

As seen in FIG. 1A, closed-ended type/format questions have a very highsatisfaction rating of 85.9%. In contrast, “Why” type/format questionshave a relatively low satisfaction rating of 56.3%. Consequently, invarious embodiments, “Why” type/format questions would be defined as lowquality question formats. In one embodiment, any question format otherthan the closed-ended type/format questions is defined as a low qualityquestion format.

In one embodiment, once low quality question formats that are predictedto result in answers that will have user satisfaction ratings below athreshold level of user satisfaction are defined/identified, low qualityquestion format identification data representing the low qualityquestion formats and format analysis instructions required to identifythe low quality question formats is generated and stored. In oneembodiment, it is stipulated that any questions determined to be in oneof the low quality question formats are defined as improperly formattedquestions.

In one embodiment, high quality question formats that are predicted toresult in answers that will have user satisfaction ratings above athreshold level of user satisfaction are defined/identified.

Returning to FIG. 1A, closed-ended type/format questions have a veryhigh satisfaction rating of 85.9%. In contrast, “Why” type/formatquestions have a relatively low satisfaction rating of 56.3%.Consequently, in various embodiments, closed-ended type/format questionswould be defined as high quality questions.

In one embodiment, once high quality question formats that are predictedto result in answers that will have user satisfaction ratings above athreshold level of user satisfaction are defined/identified, highquality question format identification data representing the highquality question formats and question format analysis instructionsrequired to identify the high quality question formats is generated andstored. In one embodiment, it is stipulated that any questionsdetermined to be in one of the high quality question formats areproperly formatted questions.

As noted above, closed-ended type/format questions have a very highsatisfaction rating of 85.9%. Consequently, in various embodiments, allquestions of the closed-ended type/format are defined as properlyformatted questions and any question in a question format other than the“closed ended” format is defined as an improperly formatted question.

Those of skill in the art will recognize that it is not necessary todefine both high quality question formats and low quality questionformats, or properly formatted and improperly formatted questions, in asingle embodiment since definition of either high quality questionformats/properly formatted questions or low quality questionformats/improperly formatted questions will, by default, define allother question formats as low quality question formats/improperlyformatted questions or high quality question formats/properly formattedquestions. Consequently, the discussion above is directed to only oneillustrative embodiment.

As discussed above, in one embodiment, users of the software system areprovided the capability to submit questions regarding the installation,implementation, use and operation of the software system through thequestion and answer based customer support system. In one embodiment, auser of the software system begins to enter, and/or submit, questiondata through the question and answer based customer support system andat least a portion of the question data is received by the question andanswer based customer support system.

In one embodiment, questions submitted to the question and answer basedcustomer support system by asking users, i.e., users submitting newquestions, are meant to be answered by members of the support communityof customer support personnel.

In various embodiments, the users of the software system enter questiondata through a question submission user interface provided through thequestion and answer based customer support system in the form of textdata, audio data, symbolic data, and/or any other means, mechanism,process, or system, for entering question data as discussed herein,and/or as known in the art at the time of filing, and/or as developedafter the time of filing.

As noted above, questions submitted to the question and answer basedcustomer support system can be formulated in a variety of ways, directedto various broad categories/subjects, and in be submitted in variousquestion formats representing various question types.

In one embodiment, as the question data is being entered by a user,and/or is otherwise received, the question data is analyzed beforeproviding the question data to any of the one or more support personnel.As noted above, in one embodiment, questions submitted to the questionand answer based customer support system by asking users are meant to beanswered by members of the support community of customer supportpersonnel. However, using the methods and systems disclosed herein, thequestion data is analyzed before providing the question data to any ofthe one or more support personnel to determine if the question datarepresents an improperly formatted question.

In one embodiment, as the question data is being entered and/orsubmitted, the question data is analyzed using the format analysisinstructions required to identify the low quality question formatsgenerated and stored as described above. In another embodiment, as thequestion data is being entered and/or, the question data is analyzedusing the format analysis instructions required to identify the highquality question formats generated and stored as discussed above.

In one embodiment, general knowledge/open-ended category questionssubmitted are identified. As noted above, general knowledge/open-endedcategory questions are of the form “Who,” “What,” “Where,” “When,”“How,” and “Why” formatted questions. Consequently, in one embodiment,the question data is analyzed to detect these terms, or their functionalequivalents.

In one embodiment, “How” and “Why” question formats are detectable usingformat analysis instructions that implement regular expressionsanalysis. Similarly “When”, “Where” and “Who” question types aredetectable using format analysis instructions that implement regularexpressions analysis, however, the analysis is slightly more involved asit typically requires part-of-speech tagging to avoid confusion withrelative adverbs and relative pronouns.

In one embodiment, closed-ended category questions submitted areidentified. In general, closed-ended question formats are detectableusing format analysis instructions that implement regular expressionsanalysis.

As noted above, most closed-ended category format questions are in thesub-category of “Yes-No” type questions. These “Yes-No” type questionsare identified by the fact that they typically start with an auxiliaryverb such as “Do”, “Can”, “Be.” As also noted above, the secondsub-category of closed-ended question format includes “Choice” typequestions. These “Choice” type questions are identified by the fact thatthey generally start with an auxiliary verb and also contain theconjunction “or.”

In one embodiment, if a question submitted to the question and answerbased customer support system is deemed to be neither a generalknowledge/open-ended nor a closed-ended category question, then thequestion is classified as being in the ill-formed question category bydefault. Most of the ill-formed category questions do not follow agrammatically correct question format either intentionally (search querytype) or unintentionally (wrong grammar, punctuation, etc.) and are moredifficult to answer.

In one embodiment, based on the analysis of the entered question datausing the format analysis instructions required to identify the lowquality question formats generated and stored as discussed above, and/orthe format analysis instructions required to identify the high qualityquestion formats generated and stored as discussed above, the questionrepresented by the question data submitted is determined to be either aproperly formatted question or an improperly formatted question. In oneembodiment, if the analysis discussed above determines that the questiondata submitted represents an improperly formatted question, one or morecorrective actions are taken.

In various embodiments, the one or more corrective actions takeninclude, but are not limited to, filtering out the improperly formattedquestions before the improperly formatted questions areforwarded/provided to the support community, and before any resourcesare devoted to answering the improperly formatted questions.

In various embodiments, the one or more corrective actions takeninclude, but are not limited to, avoiding the improperly formattedquestions completely by refusing to accept submission of the improperlyformatted questions.

In various embodiments, the one or more corrective actions takeninclude, but are not limited to, attempting to correct the improperlyformatted questions by providing the user with a set of questiontransformation instructions used to transform an improperly formattedquestion into a properly formatted question. In one embodiment, the useris provided the format transformation instructions representingsuggestions on how to re-phrase/reform the improperly formatted questionthat are customized to the specific question data being submitted, inrelative real-time. As a result, improperly formatted questions havinglow quality question formats are transformed into properly formattedquestions having high quality question formats before the question issubmitted for response, and before any resources are devoted to actuallytrying to answer the improperly formatted question.

As a specific illustrative example, in one embodiment, an asking user'squestion data is analyzed as it is being entered into the question andanswer based customer support system. If the question is determined tobe an improperly formatted question because the question is a generalknowledge/open-ended format question, then the asking user is providedformat transformation instructions that guide the user through astep-by-step process to transform the identified generalknowledge/open-ended format question into a properly formatted questionhaving a high quality question format, such as, for example, aclosed-ended question format, capable of being answered with a simple“yes” or “no”, or a closed-ended question format capable of beinganswered via multi-choice, or mapping. In one embodiment, this thestep-by-step transformation process implemented using the formattransformation instructions is performed before the question issubmitted to the question and answer based customer support system forresponse, and before any resources are devoted to actually trying toanswer the improperly formatted question.

As another specific illustrative example, in one embodiment, an askinguser's question data is analyzed as it is being entered into thequestion and answer based customer support system. If the question isdetermined to be an improperly formatted question because the questionis in a low quality rhetorical, or an otherwise “unanswerable”, questionformat, then the asking user is provided format transformationinstructions that guide the user through a step-by-step process totransform the identified rhetorical, or unanswerable, improperlyformatted question into a properly formatted question in a high qualityquestion format, such as, for example, a closed-ended question, capableof being answered with a simple “yes” or “no”, or a closed-endedquestion capable of being answered by multi-choice, or mapping. In oneembodiment, the format transformation instructions are used to implementthe step-by-step transformation process before the question is submittedto the question and answer based customer support system for response,and before any resources are actually devoted to trying to answer theimproperly formatted question.

As another specific illustrative example, in one embodiment, an askinguser's question data is analyzed as it is being entered into thequestion and answer based customer support system. If the question isdetermined to be an improperly formatted question because the questionor query is determined to be in a low quality grammatically incorrectquestion or search query format, then the asking or searching user isprovided format transformation instructions that guide the user througha step-by-step process to transform the improperly formatted questioninto a properly formatted question having a high quality grammaticallycorrect format. In one embodiment, this the step-by-step transformationprocess is implemented using the format transformation instructionsbefore the question or query is even submitted for response, and againbefore any resources are devoted to trying to answer the improperlyformatted question.

In various embodiments, the format transformation instructions areprovided to the user through one or more question asking experiencequestion transformation interface screens used to direct users towardstransforming improperly formatted questions into properly formattedclosed-ended questions. Three illustrative examples of question-askingexperience question transformation interface screens are shown in FIGS.2A, 2B, and 2C and discussed separately below.

Using the method and system for pro-active detection and correction oflow quality questions submitted to a question and answer based customersupport system discussed herein, satisfaction with answers to questionsthat may eventually be provided through a question and answer basedcustomer support system can be predicted before the questions areformally submitted to the question and answer based customer supportsystem and/or channeled to the support community for analysis andanswering. Therefore, the concepts disclosed herein provide anopportunity to intervene in the question drafting process, in relativereal time, while the question is still being formulated, and before anyresources are devoted to actually trying to answer improperly formatted,i.e., low quality, questions. Consequently, in one embodiment, the useris coached during the user's question formulation, i.e., during user'sentry of the question data representing the question, in such a way thatthere is a significantly higher likelihood that not only the asking userwill be satisfied with the answer eventually provided, but that othersearching users accessing the question and answer pair through aquestion and answer database will also be satisfied with the answereventually provided. Therefore, the disclosed method and system forpro-active detection and correction of low quality questions submittedto a question and answer based customer support system also provides forsignificant improvements to the technical fields of customer support,information dissemination, software implementation, and user experience.

In addition, using the disclosed method and system for pro-activedetection and correction of low quality questions submitted to aquestion and answer based customer support system results in moreefficient use of human and non-human resources, fewer processor cyclesbeing utilized, reduced memory utilization, and less communicationsbandwidth being utilized to relay data to and from backend systems. As aresult, computing systems are transformed into faster, more efficient,and more effective computing systems by implementing the method andsystem for pro-active detection and correction of low quality questionssubmitted to a question and answer based customer support systemdisclosed herein.

Process

In accordance with one embodiment, a process for pro-active detectionand correction of low quality questions submitted to a question andanswer based customer support system includes providing a softwaresystem to one or more users. In one embodiment, the users of thesoftware system are also provided a question and answer based customersupport system through which question data can be entered by the users.In one embodiment, the question data entered by the users representsquestions to be provided to one or more support personnel associatedwith the question and answer based customer support system. In oneembodiment, the question data is submitted by the users so that thequestions represented by the question data can be answered by at leastone of the one or more support personnel.

In one embodiment, low quality question formats that are predicted toresult in answers that will have user satisfaction ratings below athreshold level of user satisfaction are defined/identified. In oneembodiment, any identified questions submitted in any of the low qualityquestion formats are defined/labeled as improperly formatted questions.

In one embodiment, high quality question formats that are predicted toresult in answers that will have user satisfaction ratings above athreshold level of user satisfaction are defined/identified. In oneembodiment, any questions submitted in any of the high quality questionformats are defined/labeled as properly formatted questions.

In one embodiment, when question data representing a question submittedby a user through the question and answer based customer support systemis being entered by a user, and/or is otherwise received by the questionand answer based customer support system, the question data is analyzedbefore providing the question data to any of the one or more supportpersonnel to answer to the question represented by the question data todetermine if the question data represents an improperly formattedquestion.

In one embodiment, if, based on the analysis of the question data, adetermination is made that the question data represents an improperlyformatted question, one or more corrective actions are taken beforeproviding the question data to the one or more support personnel toanswer the question represented by the question data.

Consequently, using the process for pro-active detection and correctionof low quality questions submitted to a question and answer basedcustomer support system disclosed herein, improperly formatted questionsare identified before the questions are submitted to support personnelfor the purpose of providing an answer to the question, and before anyresources are expended in an attempt to answer improperly formattedquestions. In addition, improperly formatted questions are identifiedbefore any users, including the asking user, are provided answers toimproperly formatted questions that are likely to result in low usersatisfaction ratings.

FIG. 3 is a flow chart representing one example of a process 300 forpro-active detection and correction of low quality questions submittedto a question and answer based customer support system in accordancewith one embodiment.

As seen in FIG. 3, process 300 for pro-active detection and correctionof low quality questions submitted to a question and answer basedcustomer support system begins at ENTER OPERATION 301 and process flowproceeds to PROVIDE A SOFTWARE SYSTEM OPERATION 303.

In one embodiment, at PROVIDE A SOFTWARE SYSTEM OPERATION 303, asoftware system is provided for use by one or more users. In variousembodiments, the software system of PROVIDE A SOFTWARE SYSTEM OPERATION303 is any software system as discussed herein, and/or as known at thetime of filling, and/or as developed after the time of filing.

In one embodiment, once a software system is provided at PROVIDE ASOFTWARE SYSTEM OPERATION 303, process flow proceeds to PROVIDE USERS OFTHE SOFTWARE SYSTEM A QUESTION AND ANSWER BASED CUSTOMER SUPPORT SYSTEMOPERATION 305.

In one embodiment, at PROVIDE USERS OF THE SOFTWARE SYSTEM A QUESTIONAND ANSWER BASED CUSTOMER SUPPORT SYSTEM OPERATION 305 a question andanswer based customer support system, e.g., a social question-and-answer(Q&A) system, is provided to support customers/users of the softwaresystem of PROVIDE A SOFTWARE SYSTEM OPERATION 303.

In one embodiment, the question and answer based customer support systemof PROVIDE USERS OF THE SOFTWARE SYSTEM A QUESTION AND ANSWER BASEDCUSTOMER SUPPORT SYSTEM OPERATION 305 serves as a discussion forum in anarea of endeavor related to the software system. As an illustrativeexample, in one embodiment, the question and answer based customersupport system is provided at PROVIDE USERS OF THE SOFTWARE SYSTEM AQUESTION AND ANSWER BASED CUSTOMER SUPPORT SYSTEM OPERATION 305 tosupport a tax preparation software system and therefore the discussionforum is related to “federal and state taxation and tax preparation.”

In one embodiment, once a question and answer based customer supportsystem, is provided to support customers/users of the software system ofPROVIDE A SOFTWARE SYSTEM OPERATION 303 at PROVIDE USERS OF THE SOFTWARESYSTEM A QUESTION AND ANSWER BASED CUSTOMER SUPPORT SYSTEM OPERATION305. process flow proceeds to PROVIDE THE USERS THE CAPABILITY TO SUBMITQUESTION DATA THROUGH THE QUESTION AND ANSWER BASED CUSTOMER SUPPORTSYSTEM, THE QUESTIONS REPRESENTED BY THE QUESTION DATA TO BE ANSWERED BYONE OR MORE SUPPORT PERSONNEL OPERATION 307.

In one embodiment, at PROVIDE THE USERS THE CAPABILITY TO SUBMITQUESTION DATA THROUGH THE QUESTION AND ANSWER BASED CUSTOMER SUPPORTSYSTEM, THE QUESTIONS REPRESENTED BY THE QUESTION DATA TO BE ANSWERED BYONE OR MORE SUPPORT PERSONNEL OPERATION 307 users of the software systemare provided the capability to submit questions regarding theinstallation, implementation, use and operation of the software systemof PROVIDE A SOFTWARE SYSTEM OPERATION 303 through the question andanswer based customer support system of PROVIDE USERS OF THE SOFTWARESYSTEM A QUESTION AND ANSWER BASED CUSTOMER SUPPORT SYSTEM OPERATION305.

In one embodiment, at PROVIDE THE USERS THE CAPABILITY TO SUBMITQUESTION DATA THROUGH THE QUESTION AND ANSWER BASED CUSTOMER SUPPORTSYSTEM, THE QUESTIONS REPRESENTED BY THE QUESTION DATA TO BE ANSWERED BYONE OR MORE SUPPORT PERSONNEL OPERATION 307 through the question andanswer based customer support system of PROVIDE USERS OF THE SOFTWARESYSTEM A QUESTION AND ANSWER BASED CUSTOMER SUPPORT SYSTEM OPERATION305, users of the software system of PROVIDE A SOFTWARE SYSTEM OPERATION303 are provided the capability to submit questions regarding theinstallation, implementation, use and operation of the software system.

In one embodiment, the question and answer based customer support systemof PROVIDE USERS OF THE SOFTWARE SYSTEM A QUESTION AND ANSWER BASEDCUSTOMER SUPPORT SYSTEM OPERATION 305 includes a support community ofcustomer support personnel. In one embodiment, the customer supportpersonnel include paid support personnel in the employ of the providerof the software system and non-paid volunteer expert users of thesoftware system. In one embodiment, the volunteer expert users of thesoftware system are identified and certified by the provider of thesoftware system.

In one embodiment, at PROVIDE THE USERS THE CAPABILITY TO SUBMITQUESTION DATA THROUGH THE QUESTION AND ANSWER BASED CUSTOMER SUPPORTSYSTEM, THE QUESTIONS REPRESENTED BY THE QUESTION DATA TO BE ANSWERED BYONE OR MORE SUPPORT PERSONNEL OPERATION 307, through the question andanswer based customer support system of PROVIDE USERS OF THE SOFTWARESYSTEM A QUESTION AND ANSWER BASED CUSTOMER SUPPORT SYSTEM OPERATION305, users of the software system of PROVIDE A SOFTWARE SYSTEM OPERATION303 are provided the capability to submit questions to members of thesupport community of customer support personnel.

In one embodiment, questions submitted to the question and answer basedcustomer support system of PROVIDE USERS OF THE SOFTWARE SYSTEM AQUESTION AND ANSWER BASED CUSTOMER SUPPORT SYSTEM OPERATION 305 can berelated to very different broad categories, be of various questiontypes, and be formatted in various different ways.

For example, some questions submitted to the question and answer basedcustomer support system of PROVIDE USERS OF THE SOFTWARE SYSTEM AQUESTION AND ANSWER BASED CUSTOMER SUPPORT SYSTEM OPERATION 305 areproduct related questions, e.g., questions related to pricing,installation, version choice, etc. for the software systems that oftenhave little or no relation to the subject matter/job of the softwaresystem, i.e., the endeavor supported by the software system. On theother hand, some questions submitted to the question and answer basedcustomer support system of PROVIDE USERS OF THE SOFTWARE SYSTEM AQUESTION AND ANSWER BASED CUSTOMER SUPPORT SYSTEM OPERATION 305 aresubject matter related, or substantive, questions directly related tothe subject matter/endeavor of the software system.

In general, product related questions are best answered by paid supportpersonnel in the employ of the provider of the software system whilesubject matter related questions are often best answered by volunteerexpert users of the software system. Consequently, in one embodiment, itis desirable to identify the broad category of the questions, e.g.,product related questions and subject matter related questions, androute the questions accordingly either to support personnel or volunteerexpert users of the software system.

In one embodiment, the question and answer based customer support systemof PROVIDE USERS OF THE SOFTWARE SYSTEM A QUESTION AND ANSWER BASEDCUSTOMER SUPPORT SYSTEM OPERATION 305 is used to generate reusablecontent for the question and answer based customer support system, e.g.,the question and answer based customer support system of PROVIDE USERSOF THE SOFTWARE SYSTEM A QUESTION AND ANSWER BASED CUSTOMER SUPPORTSYSTEM OPERATION 305 is used to generate a customer support question andanswer database. In one embodiment, the creation of the customer supportquestion and answer database is the primary goal of the question andanswer based customer support system of PROVIDE USERS OF THE SOFTWARESYSTEM A QUESTION AND ANSWER BASED CUSTOMER SUPPORT SYSTEM OPERATION305. This is because it has been empirically demonstrated that only 1.5%of users of a typical question and answer based customer support systemare “asking users” who submit their own questions, while the remaining98.5% of users are “searching users” who look for answers by searchingfor similar topics and questions answered in the customer supportquestion and answer database.

As discussed below in more detail, questions submitted to the questionand answer based customer support system of PROVIDE USERS OF THESOFTWARE SYSTEM A QUESTION AND ANSWER BASED CUSTOMER SUPPORT SYSTEMOPERATION 305 can also be structured/formatted in a variety of ways andthese various question formats can vary significantly in quality, andmore importantly, in the predicted user satisfaction with an answer, anyanswer, to the question.

As a specific illustrative example, questions submitted to the questionand answer based customer support system of PROVIDE USERS OF THESOFTWARE SYSTEM A QUESTION AND ANSWER BASED CUSTOMER SUPPORT SYSTEMOPERATION 305 can be, but are not limited to: generalknowledge/open-ended type questions, defined as “Who” type/formatquestions, “What” type/format questions, “When” type/format questions,“Where” type/format questions, “Why” type/format questions, and “How”type/format questions; rhetorical, or otherwise “unanswerable”questions; grammatically incorrect questions and/or queries; otherwiseill-formed questions; and/or closed-ended questions, capable of beinganswered with a simple “yes” or “no”, or via a multi-choice, or mapping.

As discussed below, each of these question structures is associated witha calculated predictability that the answer to the question, whateverthat answer may be, will be found satisfactory by the users, e.g., theasking user and/or searching users. As also discussed below, thisdiscovery by the inventors is leveraged by process 300 for pro-activedetection and correction of low quality questions submitted to aquestion and answer based customer support system to predict usersatisfaction with answers that may eventually be provided through thequestion and answer based customer support system by performingpre-submission analysis of the type/format, and other attributes, of thequestion itself, rather than post question submission analysis of ananswer provided in response to the question. This paradigm shiftingapproach to predicting user satisfaction with an answer based on theuser's question alone, and before the answer is generated, is in directcontrast to prior assumptions and approaches that focused on the answersprovided through a question and answer based customer support system,and analysis performed after both the question and answer had alreadybeen formulated and provided to users.

In one embodiment, once users of the software system are provided thecapability to submit questions regarding the installation,implementation, use and operation of the software system of PROVIDE ASOFTWARE SYSTEM OPERATION 303 through the question and answer basedcustomer support system of PROVIDE USERS OF THE SOFTWARE SYSTEM AQUESTION AND ANSWER BASED CUSTOMER SUPPORT SYSTEM OPERATION 305 atPROVIDE THE USERS THE CAPABILITY TO SUBMIT QUESTION DATA THROUGH THEQUESTION AND ANSWER BASED CUSTOMER SUPPORT SYSTEM, THE QUESTIONSREPRESENTED BY THE QUESTION DATA TO BE ANSWERED BY ONE OR MORE SUPPORTPERSONNEL OPERATION 307, process flow proceeds to DEFINE LOW QUALITYQUESTION FORMATS THAT ARE PREDICTED TO RESULT IN ANSWERS THAT WILL HAVEUSER SATISFACTION RATINGS BELOW A THRESHOLD LEVEL OF USER SATISFACTIONOPERATION 309.

In one embodiment, at DEFINE LOW QUALITY QUESTION FORMATS THAT AREPREDICTED TO RESULT IN ANSWERS THAT WILL HAVE USER SATISFACTION RATINGSBELOW A THRESHOLD LEVEL OF USER SATISFACTION OPERATION 309, low qualityquestion formats that are predicted to result in answers that will haveuser satisfaction ratings below a threshold level of user satisfactionare defined/identified.

As discussed above, the embodiments disclosed herein were developed toincorporate theories and address relationships discovered throughanalysis of data collected from a specific question and answer basedcustomer support system providing support for a specific softwaresystem. This data was then leveraged to develop process 300 forpro-active detection and correction of low quality questions submittedto a question and answer based customer support system.

As noted above, questions submitted to the question and answer basedcustomer support system can be formulated in a variety of ways anddirected to various broad categories/subjects, such as “product relatedquestions”, e.g., questions related to pricing, installation, versionchoice, etc. of the software system that have little or no relation tothe subject matter/endeavor supported by the software system and“subject matter related,” or substantive questions, directly related tothe subject matter/endeavor supported by the software system. As alsonoted above, questions submitted to question and answer based customersupport system can be submitted in a variety question types/formats.Consequently, in one embodiment, process 300 for pro-active detectionand correction of low quality questions submitted to a question andanswer based customer support system is based on an empiric questiontaxonomy taking into account the question format, the question grammarstructure, the type of anticipated answer, asker type, and various otherfactors.

In one embodiment, closed-ended category questions areidentified/defined. Most closed-ended formatted questions are placed inthe sub-category of “Yes-No” type questions. These “Yes-No” typequestions typically start with an auxiliary verb such as “Do”, “Can”,“Be.” As indicated by the name, the “Yes-No” type questions can beanswered by a “Yes” or “No” answer. A specific illustrative example of a“Yes-No” questions would be: “Can I deduct my computer?” with thepossible answers “Yes, you can” or “No you can't.”

The second sub-category of closed-ended question format includes“Choice” type questions. “Choice” type questions generally start with anauxiliary verb as well, but also contain the conjunction “or.”Consequently, “Choice” type questions usually result in a multiplechoice answer embedded in the body of the question. A specificillustrative example of a “Choice” type question would be “Should Iinstall X or Y version?” with the possible answers “You need to installY,” “You need to install X,” “You need to install either X or Y,” “Youneed to install neither X or Y.”

As seen in FIG. 1A, closed-ended questions result in an impressive 85.9%up vote, i.e., 85.9% of users who submitted questions in theclosed-ended format were satisfied with the answer provided. As seen inFIG. 1A this was the highest satisfaction rating of all questionformats.

In one embodiment, general knowledge/open-ended category questions areidentified/defined including “Who” type/format questions, “What”type/format questions, “When” type/format questions, “Where” type/formatquestions, “Why” type/format questions, and “How” type/format questions.“How” and “Why” question formats are detectable with format analysisinstructions that implement regular expressions analysis. Similarly“When”, “Where” and “Who” question formats are detectable with formatanalysis instructions that implement regular expressions analysis butthe analysis is slightly more involved as it typically requirespart-of-speech tagging to avoid confusion with relative adverbs andrelative pronouns. However, as seen in FIG. 1A these question types areless common in question and answer based customer support systems. Moreexotic question types such as “tag questions,” “leading questions,” and“embedded questions” are extremely rare in question and answer basedcustomer support systems, and therefore are largely excluded fromanalysis.

In one embodiment, if a question submitted to the question and answerbased customer support system is deemed to be neither a generalknowledge/open-ended nor a closed-ended format question, the question isclassified as being in the ill-formed question category by default. Mostof the ill-formed category questions do not follow a grammaticallycorrect question format either intentionally, as in the case of a searchquery type, or unintentionally, e.g., wrong grammar, punctuation, etc.

In one embodiment, low quality question formats that are predicted toresult in answers that will have user satisfaction ratings below athreshold level of user satisfaction are defined/identified based on theempirical data obtained as discussed above and shown in FIG. 1A.

Returning to FIG. 1A, a table of results data obtained through theanalysis of actual questions submitted to a question and answer basedcustomer support system shows question types, also referred to herein asformats and/or categories, (column 101) of the questions received, thefrequency of the question types as a percentage of questions asked(column 102), and ranking of the question types by up vote fraction(column 103), in descending order.

As seen in FIG. 1A, observed statistics relating up and down votesassociated with the answers provided to each question type/format isshown. As seen in FIG. 1A, closed-ended type/format questions have avery high satisfaction rating of 85.9%. In contrast, “Why” type/formatquestions have a relatively low satisfaction rating of 56.3%.Consequently, in various embodiments, “Why” type/format questions wouldbe defined as low quality question formats at DEFINE LOW QUALITYQUESTION FORMATS THAT ARE PREDICTED TO RESULT IN ANSWERS THAT WILL HAVEUSER SATISFACTION RATINGS BELOW A THRESHOLD LEVEL OF USER SATISFACTIONOPERATION 309.

In one embodiment, any question format having a satisfaction rating ofless than a threshold level of user satisfaction of 85.9%, i.e., anyquestion type/format other than “closed-ended”, would be defined as lowquality question formats at DEFINE LOW QUALITY QUESTION FORMATS THAT AREPREDICTED TO RESULT IN ANSWERS THAT WILL HAVE USER SATISFACTION RATINGSBELOW A THRESHOLD LEVEL OF USER SATISFACTION OPERATION 309.

In one embodiment, any question formats having a satisfaction rating ofless than a threshold level of user satisfaction of 81.4%, i.e., anyquestion type/format other than “closed-ended” or “Who”, would bedefined as low quality question formats at DEFINE LOW QUALITY QUESTIONFORMATS THAT ARE PREDICTED TO RESULT IN ANSWERS THAT WILL HAVE USERSATISFACTION RATINGS BELOW A THRESHOLD LEVEL OF USER SATISFACTIONOPERATION 309.

In one embodiment, any question formats having a satisfaction rating ofless than a threshold level of user satisfaction of 73.1%, i.e., anyquestion type/format other than “closed-ended”, “Who”, or “What” wouldbe defined as low quality question formats at DEFINE LOW QUALITYQUESTION FORMATS THAT ARE PREDICTED TO RESULT IN ANSWERS THAT WILL HAVEUSER SATISFACTION RATINGS BELOW A THRESHOLD LEVEL OF USER SATISFACTIONOPERATION 309.

In one embodiment, any question formats having a satisfaction rating ofless than a threshold level of user satisfaction of 70.2%, i.e., anyquestion type/format other than “closed-ended”, “Who,”, “What,” or“When” would be defined as low quality question formats at DEFINE LOWQUALITY QUESTION FORMATS THAT ARE PREDICTED TO RESULT IN ANSWERS THATWILL HAVE USER SATISFACTION RATINGS BELOW A THRESHOLD LEVEL OF USERSATISFACTION OPERATION 309.

In various embodiments, the threshold level of user satisfactionrequired to avoid being identified as a low quality question format canbe any threshold level as defined by the provider of process 300 forpro-active detection and correction of low quality questions submittedto a question and answer based customer support system and/or thequestion and answer based customer support system of PROVIDE USERS OFTHE SOFTWARE SYSTEM A QUESTION AND ANSWER BASED CUSTOMER SUPPORT SYSTEMOPERATION 305.

In one embodiment, once low quality question formats that are predictedto result in answers that will have user satisfaction ratings below athreshold level of user satisfaction are defined/identified at DEFINELOW QUALITY QUESTION FORMATS THAT ARE PREDICTED TO RESULT IN ANSWERSTHAT WILL HAVE USER SATISFACTION, RATINGS BELOW A THRESHOLD LEVEL OFUSER SATISFACTION OPERATION 309, low quality question formatidentification data representing the low quality question formats andformat analysis instructions required to identify the low qualityquestion formats is generated and stored.

In one embodiment, once low quality question formats that are predictedto result in answers that will have user satisfaction ratings below athreshold level of user satisfaction are defined/identified and lowquality question format identification data representing the low qualityquestion formats and format analysis instructions required to identifythe low quality question formats is generated and stored at DEFINE LOWQUALITY QUESTION FORMATS THAT ARE PREDICTED TO RESULT IN ANSWERS THATWILL HAVE USER SATISFACTION RATINGS BELOW A THRESHOLD LEVEL OF USERSATISFACTION OPERATION 309, process flow proceeds to DEFINE QUESTIONSHAVING LOW QUALITY QUESTION FORMATS AS IMPROPERLY FORMATTED QUESTIONSOPERATION 311.

In one embodiment, at DEFINE QUESTIONS HAVING LOW QUALITY QUESTIONFORMATS AS IMPROPERLY FORMATTED QUESTIONS OPERATION 311, it isstipulated that any questions determined to be in one of the low qualityquestion formats of DEFINE LOW QUALITY QUESTION FORMATS THAT AREPREDICTED TO RESULT IN ANSWERS THAT WILL HAVE USER SATISFACTION RATINGSBELOW A THRESHOLD LEVEL OF USER SATISFACTION OPERATION 309 are definedas improperly formatted questions.

In one embodiment, once it is stipulated that any questions determinedto be in one of the low quality question formats of DEFINE LOW QUALITYQUESTION FORMATS THAT ARE PREDICTED TO RESULT IN ANSWERS THAT WILL HAVEUSER SATISFACTION RATINGS BELOW A THRESHOLD LEVEL OF USER SATISFACTIONOPERATION 309 are improperly formatted questions at DEFINE QUESTIONSHAVING LOW QUALITY QUESTION FORMATS AS IMPROPERLY FORMATTED QUESTIONSOPERATION 311, process flow proceeds to DEFINE HIGH QUALITY QUESTIONFORMATS THAT ARE PREDICTED TO RESULT IN ANSWERS THAT WILL HAVE USERSATISFACTION RATINGS ABOVE A THRESHOLD LEVEL OF USER SATISFACTIONOPERATION 313.

In one embodiment, at DEFINE HIGH QUALITY QUESTION FORMATS THAT AREPREDICTED TO RESULT IN ANSWERS THAT WILL HAVE USER SATISFACTION RATINGSABOVE A THRESHOLD LEVEL OF USER SATISFACTION OPERATION 313, high qualityquestion formats that are predicted to result in answers that will haveuser satisfaction ratings above a threshold level of user satisfactionare defined/identified.

Returning to FIG. 1A, closed-ended type/format questions have a veryhigh satisfaction rating of 85.9%. In contrast, “Why” type/formatquestions have a relatively low satisfaction rating of 56.3%.Consequently, in various embodiments, closed-ended type/format questionswould be defined as high quality questions formats at DEFINE HIGHQUALITY QUESTION FORMATS THAT ARE PREDICTED TO RESULT IN ANSWERS THATWILL HAVE USER SATISFACTION RATINGS ABOVE A THRESHOLD LEVEL OF USERSATISFACTION OPERATION 313.

In one embodiment, any question formats having a satisfaction ratinggreater than a threshold level of user satisfaction of 82%, i.e., the“closed-ended” question format, would be defined as high qualityquestion formats at DEFINE HIGH QUALITY QUESTION FORMATS THAT AREPREDICTED TO RESULT IN ANSWERS THAT WILL HAVE USER SATISFACTION RATINGSABOVE A THRESHOLD LEVEL OF USER SATISFACTION OPERATION 313.

In one embodiment, any question formats having a satisfaction ratinggreater than a threshold level of user satisfaction of 81%, i.e., the“closed-ended” or “Who” question formats, would be defined as highquality question formats at DEFINE HIGH QUALITY QUESTION FORMATS THATARE PREDICTED TO RESULT IN ANSWERS THAT WILL HAVE USER SATISFACTIONRATINGS ABOVE A THRESHOLD LEVEL OF USER SATISFACTION OPERATION 313.

In one embodiment, any question formats having a satisfaction ratinggreater than a threshold level of user satisfaction of 73%, i.e., the“closed-ended”, “Who”, or “What” question formats would be defined ashigh quality question formats at DEFINE HIGH QUALITY QUESTION FORMATSTHAT ARE PREDICTED TO RESULT IN ANSWERS THAT WILL HAVE USER SATISFACTIONRATINGS ABOVE A THRESHOLD LEVEL OF USER SATISFACTION OPERATION 313.

In one embodiment, any question formats having a satisfaction ratinggreater than a threshold level of user satisfaction of 70%, i.e., the“closed-ended”, “Who,”, “What,” “When,” or “Where” question formatswould be defined as high quality question formats at DEFINE HIGH QUALITYQUESTION FORMATS THAT ARE PREDICTED TO RESULT IN ANSWERS THAT WILL HAVEUSER SATISFACTION RATINGS ABOVE A THRESHOLD LEVEL OF USER SATISFACTIONOPERATION 313.

In various embodiments, the threshold level of user satisfactionrequired to avoid being identified as a low quality question format canbe any threshold level as defined by the provider of process 300 forpro-active detection and correction of low quality questions submittedto a question and answer based customer support system and/or thequestion and answer based customer support system of PROVIDE USERS OFTHE SOFTWARE SYSTEM A QUESTION AND ANSWER BASED CUSTOMER SUPPORT SYSTEMOPERATION 305.

As noted above, closed-ended type/format questions have a very highsatisfaction rating of 85.9%. Consequently, in various embodiments,closed-ended type/format questions are defined as high quality questionformats at DEFINE HIGH QUALITY QUESTION FORMATS THAT ARE PREDICTED TORESULT IN ANSWERS THAT WILL HAVE USER SATISFACTION RATINGS ABOVE ATHRESHOLD LEVEL OF USER SATISFACTION OPERATION 313 and any questionformat other than “closed ended” format is defined as a low qualityquestion format at DEFINE LOW QUALITY QUESTION FORMATS THAT AREPREDICTED TO RESULT IN ANSWERS THAT WILL HAVE USER SATISFACTION, RATINGSBELOW A THRESHOLD LEVEL OF USER SATISFACTION OPERATION 309.

In one embodiment, once high quality question formats that are predictedto result in answers that will have user satisfaction ratings above athreshold level of user satisfaction are defined/identified at DEFINEHIGH QUALITY QUESTION FORMATS THAT ARE PREDICTED TO RESULT IN ANSWERSTHAT WILL HAVE USER SATISFACTION RATINGS ABOVE A THRESHOLD LEVEL OF USERSATISFACTION OPERATION 313, high quality question format identificationdata representing the high quality question formats and question formatanalysis instructions required to identify the high quality questionformats is generated and stored.

In one embodiment, once high quality question formats that are predictedto result in answers that will have user satisfaction ratings above athreshold level of user satisfaction are defined/identified and highquality question format identification data representing the highquality question formats and question format analysis instructionsrequired to identify the high quality question formats is generated andstored at DEFINE HIGH QUALITY QUESTION FORMATS THAT ARE PREDICTED TORESULT IN ANSWERS THAT WILL HAVE USER SATISFACTION RATINGS ABOVE ATHRESHOLD LEVEL OF USER SATISFACTION OPERATION 313, process flowproceeds to DEFINE QUESTIONS HAVING HIGH QUALITY QUESTION FORMATS ASPROPERLY FORMATTED QUESTIONS OPERATION 315.

In one embodiment, at DEFINE QUESTIONS HAVING HIGH QUALITY QUESTIONFORMATS AS PROPERLY FORMATTED QUESTIONS OPERATION 315, it is stipulatedthat any questions determined to be in one of the high quality questionformats that are predicted to result in answers that will have usersatisfaction ratings above a threshold level of user satisfaction ofDEFINE HIGH QUALITY QUESTION FORMATS THAT ARE PREDICTED TO RESULT INANSWERS THAT WILL HAVE USER SATISFACTION RATINGS ABOVE A THRESHOLD LEVELOF USER SATISFACTION OPERATION 313 are properly formatted questions.

As noted above, closed-ended type/format questions have a very highsatisfaction rating of 85.9%. Consequently, in various embodiments, allquestions of the closed-ended type/format are defined as properlyformatted questions at DEFINE QUESTIONS HAVING HIGH QUALITY QUESTIONFORMATS AS PROPERLY FORMATTED QUESTIONS OPERATION 315 and any questionin a question format other than the “closed ended” format is defined asan improperly formatted question at DEFINE QUESTIONS HAVING LOW QUALITYQUESTION FORMATS AS IMPROPERLY FORMATTED QUESTIONS OPERATION 311.

Those of skill in the art will recognize that it is not necessary todefine both high quality question formats and low quality questionformats, or properly formatted and improperly formatted questions, in asingle embodiment since definition of either high quality questionformats/properly formatted questions or low quality questionformats/improperly formatted questions will, by default, define allother question formats as low quality question formats/improperlyformatted questions or high quality question formats/properly formattedquestions. Consequently, the discussion above is directed to only oneillustrative embodiment.

In one embodiment, once it is stipulated that any questions determinedto be in one of the high quality question formats that are predicted toresult in answers that will have user satisfaction ratings above athreshold level of user satisfaction of DEFINE HIGH QUALITY QUESTIONFORMATS THAT ARE PREDICTED TO RESULT IN ANSWERS THAT WILL HAVE USERSATISFACTION RATINGS ABOVE A THRESHOLD LEVEL OF USER SATISFACTIONOPERATION 313 are properly formatted questions at DEFINE QUESTIONSHAVING HIGH QUALITY QUESTION FORMATS AS PROPERLY FORMATTED QUESTIONSOPERATION 315 process flow proceeds to RECEIVE QUESTION DATAREPRESENTING A QUESTION SUBMITTED BY A USER THROUGH THE QUESTION ANDANSWER BASED CUSTOMER SUPPORT SYSTEM OPERATION 317.

As discussed above, in one embodiment, users of the software system ofPROVIDE A SOFTWARE SYSTEM OPERATION 303 are provided the capability tosubmit questions regarding the installation, implementation, use andoperation of the software system through the question and answer basedcustomer support system of PROVIDE USERS OF THE SOFTWARE SYSTEM AQUESTION AND ANSWER BASED CUSTOMER SUPPORT SYSTEM OPERATION 305.

In one embodiment, at RECEIVE QUESTION DATA REPRESENTING A QUESTIONSUBMITTED BY A USER THROUGH THE QUESTION AND ANSWER BASED CUSTOMERSUPPORT SYSTEM OPERATION 317, a user of the software system of PROVIDE ASOFTWARE SYSTEM OPERATION 303 begins to enter, and/or submit, questiondata through the question and answer based customer support system ofPROVIDE USERS OF THE SOFTWARE SYSTEM A QUESTION AND ANSWER BASEDCUSTOMER SUPPORT SYSTEM OPERATION 305 and at least a portion of thequestion data is received by the question and answer based customersupport system of PROVIDE USERS OF THE SOFTWARE SYSTEM A QUESTION ANDANSWER BASED CUSTOMER SUPPORT SYSTEM OPERATION 305.

In one embodiment, questions submitted to the question and answer basedcustomer support system by asking users at RECEIVE QUESTION DATAREPRESENTING A QUESTION SUBMITTED BY A USER THROUGH THE QUESTION ANDANSWER BASED CUSTOMER SUPPORT SYSTEM OPERATION 317 are meant to beanswered by members of the support community of customer supportpersonnel.

In various embodiments, at RECEIVE QUESTION DATA REPRESENTING A QUESTIONSUBMITTED BY A USER THROUGH THE QUESTION AND ANSWER BASED CUSTOMERSUPPORT SYSTEM OPERATION 317 the users of the software system enterquestion data through a question submission user interface providedthrough the question and answer based customer support system in theform of text data, audio data, symbolic data, and/or any other means,mechanism, process, or system, for entering question data as discussedherein, and/or as known in the art at the time of filing, and/or asdeveloped after the time of filing.

As noted above, questions submitted to the question and answer basedcustomer support system at RECEIVE QUESTION DATA REPRESENTING A QUESTIONSUBMITTED BY A USER THROUGH THE QUESTION AND ANSWER BASED CUSTOMERSUPPORT SYSTEM OPERATION 317 can be formulated in a variety of ways anddirected to various broad categories/subjects and in various questionformats representing various question types.

In one embodiment, once a user of the software system begins to enter,and/or submit, question data through the question and answer basedcustomer support system, and at least a portion of the question data isreceived by the question and answer based customer support system, atRECEIVE QUESTION DATA REPRESENTING A QUESTION SUBMITTED BY A USERTHROUGH THE QUESTION AND ANSWER BASED CUSTOMER SUPPORT SYSTEM OPERATION317, process flow proceeds to BEFORE PROVIDING THE QUESTION DATA TO ANYOF THE ONE OR MORE SUPPORT PERSONNEL FOR THE PURPOSE OF PROVIDING ANANSWER TO THE QUESTION, ANALYZE THE QUESTION DATA TO DETERMINE IF THEQUESTION DATA REPRESENTS AN IMPROPERLY FORMATTED QUESTION OPERATION 319.

In one embodiment, at BEFORE PROVIDING THE QUESTION DATA TO ANY OF THEONE OR MORE SUPPORT PERSONNEL FOR THE PURPOSE OF PROVIDING AN ANSWER TOTHE QUESTION, ANALYZE THE QUESTION DATA TO DETERMINE IF THE QUESTIONDATA REPRESENTS AN IMPROPERLY FORMATTED QUESTION OPERATION 319, as thequestion data is being entered by a user, and/or is otherwise received,at RECEIVE QUESTION DATA REPRESENTING A QUESTION SUBMITTED BY A USERTHROUGH THE QUESTION AND ANSWER BASED CUSTOMER SUPPORT SYSTEM OPERATION317, the question data is analyzed before providing the question data toany of the one or more support personnel.

As noted above, in one embodiment, questions submitted to the questionand answer based customer support system by asking users at RECEIVEQUESTION DATA REPRESENTING A QUESTION SUBMITTED BY A USER THROUGH THEQUESTION AND ANSWER BASED CUSTOMER SUPPORT SYSTEM OPERATION 317 aremeant to be answered by members of the support community of customersupport personnel. However, at BEFORE PROVIDING THE QUESTION DATA TO ANYOF THE ONE OR MORE SUPPORT PERSONNEL FOR THE PURPOSE OF PROVIDING ANANSWER TO THE QUESTION, ANALYZE THE QUESTION DATA TO DETERMINE IF THEQUESTION DATA REPRESENTS AN IMPROPERLY FORMATTED QUESTION OPERATION 319the question data is analyzed before providing the question data to anyof the one or more support personnel to determine if the question datarepresents an improperly formatted question as defined at DEFINEQUESTIONS HAVING LOW QUALITY QUESTION FORMATS AS IMPROPERLY FORMATTEDQUESTIONS OPERATION 311 and/or DEFINE HIGH QUALITY QUESTION FORMATS THATARE PREDICTED TO RESULT IN ANSWERS THAT WILL HAVE USER SATISFACTIONRATINGS ABOVE A THRESHOLD LEVEL OF USER SATISFACTION OPERATION 313.

In one embodiment, as the question data is being entered and/orsubmitted at RECEIVE QUESTION DATA REPRESENTING A QUESTION SUBMITTED BYA USER THROUGH THE QUESTION AND ANSWER BASED CUSTOMER SUPPORT SYSTEMOPERATION 317, the question data is analyzed using the format analysisinstructions required to identify the low quality question formatsgenerated and stored at DEFINE LOW QUALITY QUESTION FORMATS THAT AREPREDICTED TO RESULT IN ANSWERS THAT WILL HAVE USER SATISFACTION, RATINGSBELOW A THRESHOLD LEVEL OF USER SATISFACTION OPERATION 309.

In one embodiment, as the question data is being entered and/orsubmitted at RECEIVE QUESTION DATA REPRESENTING A QUESTION SUBMITTED BYA USER THROUGH THE QUESTION AND ANSWER BASED CUSTOMER SUPPORT SYSTEMOPERATION 317, the question data is analyzed using the format analysisinstructions required to identify the high quality question formatsgenerated and stored at DEFINE HIGH QUALITY QUESTION FORMATS THAT AREPREDICTED TO RESULT IN ANSWERS THAT WILL HAVE USER SATISFACTION RATINGSABOVE A THRESHOLD LEVEL OF USER SATISFACTION OPERATION 313.

In one embodiment, general knowledge/open-ended category questionssubmitted at RECEIVE QUESTION DATA REPRESENTING A QUESTION SUBMITTED BYA USER THROUGH THE QUESTION AND ANSWER BASED CUSTOMER SUPPORT SYSTEMOPERATION 317 are identified at BEFORE PROVIDING THE QUESTION DATA TOANY OF THE ONE OR MORE SUPPORT PERSONNEL FOR THE PURPOSE OF PROVIDING ANANSWER TO THE QUESTION, ANALYZE THE QUESTION DATA TO DETERMINE IF THEQUESTION DATA REPRESENTS AN IMPROPERLY FORMATTED QUESTION OPERATION 319.

As noted above, general knowledge/open-ended category questions are ofthe form “Who,” “What,” “Where,” “When,” “How,” and “Why: formattedquestions. Consequently, at BEFORE PROVIDING THE QUESTION DATA TO ANY OFTHE ONE OR MORE SUPPORT PERSONNEL FOR THE PURPOSE OF PROVIDING AN ANSWERTO THE QUESTION, ANALYZE THE QUESTION DATA TO DETERMINE IF THE QUESTIONDATA REPRESENTS AN IMPROPERLY FORMATTED QUESTION OPERATION 319 thequestion data is analyzed to detect these terms, or their functionalequivalents.

“How” and “Why” question formats are detectable at BEFORE PROVIDING THEQUESTION DATA TO ANY OF THE ONE OR MORE SUPPORT PERSONNEL FOR THEPURPOSE OF PROVIDING AN ANSWER TO THE QUESTION, ANALYZE THE QUESTIONDATA TO DETERMINE IF THE QUESTION DATA REPRESENTS AN IMPROPERLYFORMATTED QUESTION OPERATION 319 using format analysis instructions thatimplement regular expressions analysis. Similarly “When”, “Where” and“Who” question types are detectable at BEFORE PROVIDING THE QUESTIONDATA TO ANY OF THE ONE OR MORE SUPPORT PERSONNEL FOR THE PURPOSE OFPROVIDING AN ANSWER TO THE QUESTION, ANALYZE THE QUESTION DATA TODETERMINE IF THE QUESTION DATA REPRESENTS AN IMPROPERLY FORMATTEDQUESTION OPERATION 319 using format analysis instructions that implementregular expressions analysis but the analysis is slightly more involvedas it typically requires part-of-speech tagging to avoid confusion withrelative adverbs and relative pronouns.

In one embodiment, closed-ended category questions submitted at RECEIVEQUESTION DATA REPRESENTING A QUESTION SUBMITTED BY A USER THROUGH THEQUESTION AND ANSWER BASED CUSTOMER SUPPORT SYSTEM OPERATION 317 areidentified at BEFORE PROVIDING THE QUESTION DATA TO ANY OF THE ONE ORMORE SUPPORT PERSONNEL FOR THE PURPOSE OF PROVIDING AN ANSWER TO THEQUESTION, ANALYZE THE QUESTION DATA TO DETERMINE IF THE QUESTION DATAREPRESENTS AN IMPROPERLY FORMATTED QUESTION OPERATION 319.

In general, closed-ended question formats are detectable at BEFOREPROVIDING THE QUESTION DATA TO ANY OF THE ONE OR MORE SUPPORT PERSONNELFOR THE PURPOSE OF PROVIDING AN ANSWER TO THE QUESTION, ANALYZE THEQUESTION DATA TO DETERMINE IF THE QUESTION DATA REPRESENTS AN IMPROPERLYFORMATTED QUESTION OPERATION 319 using format analysis instructions thatimplement regular expressions analysis.

As noted above, most closed-ended category format questions are in thesub-category of “Yes-No” type questions. These “Yes-No” type questionsare identified at BEFORE PROVIDING THE QUESTION DATA TO ANY OF THE ONEOR MORE SUPPORT PERSONNEL FOR THE PURPOSE OF PROVIDING AN ANSWER TO THEQUESTION, ANALYZE THE QUESTION DATA TO DETERMINE IF THE QUESTION DATAREPRESENTS AN IMPROPERLY FORMATTED QUESTION OPERATION 319 by the factthat they typically start with an auxiliary verb such as “Do”, “Can”,“Be.”

As noted above, the second sub-category of closed-ended question formatincludes “Choice” type questions. These “Choice” type questions areidentified at BEFORE PROVIDING THE QUESTION DATA TO ANY OF THE ONE ORMORE SUPPORT PERSONNEL FOR THE PURPOSE OF PROVIDING AN ANSWER TO THEQUESTION, ANALYZE THE QUESTION DATA TO DETERMINE IF THE QUESTION DATAREPRESENTS AN IMPROPERLY FORMATTED QUESTION OPERATION 319 by the factthat they generally start with an auxiliary verb as well, and alsocontain the conjunction “or.”

In one embodiment, if a question submitted to the question and answerbased customer support system at RECEIVE QUESTION DATA REPRESENTING AQUESTION SUBMITTED BY A USER THROUGH THE QUESTION AND ANSWER BASEDCUSTOMER SUPPORT SYSTEM OPERATION 317 is deemed to be neither a generalknowledge/open-ended nor a closed-ended category question at BEFOREPROVIDING THE QUESTION DATA TO ANY OF THE ONE OR MORE SUPPORT PERSONNELFOR THE PURPOSE OF PROVIDING AN ANSWER TO THE QUESTION, ANALYZE THEQUESTION DATA TO DETERMINE IF THE QUESTION DATA REPRESENTS AN IMPROPERLYFORMATTED QUESTION OPERATION 319, then the question is classified asbeing in the ill-formed question category by default. Most of theill-formed category questions do not follow a grammatically correctquestion format either intentionally (search query type) orunintentionally (wrong grammar, punctuation, etc.) and are moredifficult to answer.

In one embodiment, based on the analysis of the entered question datausing the format analysis instructions required to identify the lowquality question formats generated and stored at DEFINE LOW QUALITYQUESTION FORMATS THAT ARE PREDICTED TO RESULT IN ANSWERS THAT WILL HAVEUSER SATISFACTION, RATINGS BELOW A THRESHOLD LEVEL OF USER SATISFACTIONOPERATION 309 and/or the format analysis instructions required toidentify the high quality question formats generated and stored atDEFINE HIGH QUALITY QUESTION FORMATS THAT ARE PREDICTED TO RESULT INANSWERS THAT WILL HAVE USER SATISFACTION RATINGS ABOVE A THRESHOLD LEVELOF USER SATISFACTION OPERATION 313, the question represented by thequestion data submitted at RECEIVE QUESTION DATA REPRESENTING A QUESTIONSUBMITTED BY A USER THROUGH THE QUESTION AND ANSWER BASED CUSTOMERSUPPORT SYSTEM OPERATION 317 is determined to be either a properlyformatted question or an improperly formatted question at BEFOREPROVIDING THE QUESTION DATA TO ANY OF THE ONE OR MORE SUPPORT PERSONNELFOR THE PURPOSE OF PROVIDING AN ANSWER TO THE QUESTION, ANALYZE THEQUESTION DATA TO DETERMINE IF THE QUESTION DATA REPRESENTS AN IMPROPERLYFORMATTED QUESTION OPERATION 319.

As noted above, in one embodiment, questions submitted to the questionand answer based customer support system by users at RECEIVE QUESTIONDATA REPRESENTING A QUESTION SUBMITTED BY A USER THROUGH THE QUESTIONAND ANSWER BASED CUSTOMER SUPPORT SYSTEM OPERATION 317 are meant to beanswered by members of the support community of customer supportpersonnel. However, in one embodiment, at BEFORE PROVIDING THE QUESTIONDATA TO ANY OF THE ONE OR MORE SUPPORT PERSONNEL FOR THE PURPOSE OFPROVIDING AN ANSWER TO THE QUESTION, ANALYZE THE QUESTION DATA TODETERMINE IF THE QUESTION DATA REPRESENTS AN IMPROPERLY FORMATTEDQUESTION OPERATION 319, the question data is analyzed and adetermination as to whether the question data represents a properlyformatted question or an improperly formatted question is made beforeproviding the question data to any of the one or more support personnelfor the purpose of providing an answer to the question to determine ifthe question data represents an improperly formatted question.Consequently, in one embodiment, the analysis of BEFORE PROVIDING THEQUESTION DATA TO ANY OF THE ONE OR MORE SUPPORT PERSONNEL FOR THEPURPOSE OF PROVIDING AN ANSWER TO THE QUESTION, ANALYZE THE QUESTIONDATA TO DETERMINE IF THE QUESTION DATA REPRESENTS AN IMPROPERLYFORMATTED QUESTION OPERATION 319 is performed while the question isstill being formulated and before any resources are devoted to trying toanswer improperly formatted questions.

In one embodiment, once the question data is being analyzed beforeproviding the question data to any of the one or more support personnelfor the purpose of providing an answer to the question represented bythe question data at BEFORE PROVIDING THE QUESTION DATA TO ANY OF THEONE OR MORE SUPPORT PERSONNEL FOR THE PURPOSE OF PROVIDING AN ANSWER TOTHE QUESTION, ANALYZE THE QUESTION DATA TO DETERMINE IF THE QUESTIONDATA REPRESENTS AN IMPROPERLY FORMATTED QUESTION OPERATION 319, processflow proceeds to IF A DETERMINATION IS MADE THAT THE QUESTION DATAREPRESENTS AN IMPROPERLY FORMATTED QUESTION, TAKE ONE OR MORE CORRECTIVEACTIONS BEFORE PROVIDING THE QUESTION DATA TO THE ONE OR MORE SUPPORTPERSONNEL FOR THE PURPOSE OF PROVIDING AN ANSWER TO THE QUESTIONOPERATION 321.

In one embodiment, at IF A DETERMINATION IS MADE THAT THE QUESTION DATAREPRESENTS AN IMPROPERLY FORMATTED QUESTION, TAKE ONE OR MORE CORRECTIVEACTIONS BEFORE PROVIDING THE QUESTION DATA TO THE ONE OR MORE SUPPORTPERSONNEL FOR THE PURPOSE OF PROVIDING AN ANSWER TO THE QUESTIONOPERATION 321, if the analysis of BEFORE PROVIDING THE QUESTION DATA TOANY OF THE ONE OR MORE SUPPORT PERSONNEL FOR THE PURPOSE OF PROVIDING ANANSWER TO THE QUESTION, ANALYZE THE QUESTION DATA TO DETERMINE IF THEQUESTION DATA REPRESENTS AN IMPROPERLY FORMATTED QUESTION OPERATION 319determines that the question data submitted at RECEIVE QUESTION DATAREPRESENTING A QUESTION SUBMITTED BY A USER THROUGH THE QUESTION ANDANSWER BASED CUSTOMER SUPPORT SYSTEM OPERATION 317 represents animproperly formatted question, one or more corrective actions are taken.

In various embodiments, the one or more corrective actions taken at IF ADETERMINATION IS MADE THAT THE QUESTION DATA REPRESENTS AN IMPROPERLYFORMATTED QUESTION, TAKE ONE OR MORE CORRECTIVE ACTIONS BEFORE PROVIDINGTHE QUESTION DATA TO THE ONE OR MORE SUPPORT PERSONNEL FOR THE PURPOSEOF PROVIDING AN ANSWER TO THE QUESTION OPERATION 321 include, but arenot limited to, filtering out the improperly formatted questions beforethe improperly formatted questions are forwarded/provided to the supportcommunity, and before any resources are devoted to answering theimproperly formatted questions.

In various embodiments, the one or more corrective actions taken at IF ADETERMINATION IS MADE THAT THE QUESTION DATA REPRESENTS AN IMPROPERLYFORMATTED QUESTION, TAKE ONE OR MORE CORRECTIVE ACTIONS BEFORE PROVIDINGTHE QUESTION DATA TO THE ONE OR MORE SUPPORT PERSONNEL FOR THE PURPOSEOF PROVIDING AN ANSWER TO THE QUESTION OPERATION 321 include, but arenot limited to, avoiding the improperly formatted questions completelyby refusing to accept submission of the improperly formatted questions.

In various embodiments, the one or more corrective actions taken at IF ADETERMINATION IS MADE THAT THE QUESTION DATA REPRESENTS AN IMPROPERLYFORMATTED QUESTION, TAKE ONE OR MORE CORRECTIVE ACTIONS BEFORE PROVIDINGTHE QUESTION DATA TO THE ONE OR MORE SUPPORT PERSONNEL FOR THE PURPOSEOF PROVIDING AN ANSWER TO THE QUESTION OPERATION 321 include, but arenot limited to, attempting to correct the improperly formatted questionsby providing the user with a set of question transformation instructionsused to transform an improperly formatted question into a properlyformatted question.

In one embodiment, at IF A DETERMINATION IS MADE THAT THE QUESTION DATAREPRESENTS AN IMPROPERLY FORMATTED QUESTION, TAKE ONE OR MORE CORRECTIVEACTIONS BEFORE PROVIDING THE QUESTION DATA TO THE ONE OR MORE SUPPORTPERSONNEL FOR THE PURPOSE OF PROVIDING AN ANSWER TO THE QUESTIONOPERATION 321 the user is provided format transformation instructionsrepresenting suggestions on how to re-phrase/reform the improperlyformatted question. In one embodiment, format transformationinstructions for transforming the improperly formatted question into aproperly formatted question are customized to the specific question databeing submitted, in relative real-time. As a result, improperlyformatted questions having low quality question formats are transformedinto properly formatted questions having high quality question formatsbefore the question is submitted for response, and before any resourcesare devoted to actually trying to answer the improperly formattedquestion.

As a specific illustrative example, in one embodiment, an asking user'squestion data is analyzed at BEFORE PROVIDING THE QUESTION DATA TO ANYOF THE ONE OR MORE SUPPORT PERSONNEL FOR THE PURPOSE OF PROVIDING ANANSWER TO THE QUESTION, ANALYZE THE QUESTION DATA TO DETERMINE IF THEQUESTION DATA REPRESENTS AN IMPROPERLY FORMATTED QUESTION OPERATION 319as it is being entered into the question and answer based customersupport system. If the question is determined to be an improperlyformatted question because the question is a generalknowledge/open-ended type/format question, then at IF A DETERMINATION ISMADE THAT THE QUESTION DATA REPRESENTS AN IMPROPERLY FORMATTED QUESTION,TAKE ONE OR MORE CORRECTIVE ACTIONS BEFORE PROVIDING THE QUESTION DATATO THE ONE OR MORE SUPPORT PERSONNEL FOR THE PURPOSE OF PROVIDING ANANSWER TO THE QUESTION OPERATION 321 the asking user is provided formattransformation instructions that guide the user through a step-by-stepprocess to transform the identified general knowledge/open-ended formatquestion into a properly formatted question having a high qualityquestion format, such as, for example, a closed-ended question format,capable of being answered with a simple “yes” or “no”, or a closed-endedquestion format capable of being answered via multi-choice, or mapping.In one embodiment, this the step-by-step transformation processimplemented using the format transformation instructions is performedbefore the question is submitted to the question and answer basedcustomer support system for response, and before any resources aredevoted to actually trying to answer the improperly formatted question.

As another specific illustrative example, in one embodiment, an askinguser's question data is analyzed at BEFORE PROVIDING THE QUESTION DATATO ANY OF THE ONE OR MORE SUPPORT PERSONNEL FOR THE PURPOSE OF PROVIDINGAN ANSWER TO THE QUESTION, ANALYZE THE QUESTION DATA TO DETERMINE IF THEQUESTION DATA REPRESENTS AN IMPROPERLY FORMATTED QUESTION OPERATION 319as it is being entered into the question and answer based customersupport system. If the question is determined to be an improperlyformatted question because the question is a generalknowledge/open-ended type/format question, then at IF A DETERMINATION ISMADE THAT THE QUESTION DATA REPRESENTS AN IMPROPERLY FORMATTED QUESTION,TAKE ONE OR MORE CORRECTIVE ACTIONS BEFORE PROVIDING THE QUESTION DATATO THE ONE OR MORE SUPPORT PERSONNEL FOR THE PURPOSE OF PROVIDING ANANSWER TO THE QUESTION OPERATION 321 the asking user is provided formattransformation instructions that guide the user through a step-by-stepprocess to transform the identified general knowledge/open-ended formatquestion into the most highly rated general knowledge/open-ended formatin order of effectiveness (see FIG. 1A), i.e.: “Where” type/formatquestions, “What” type/format questions, “When” type/format questions,“Who” type/format questions, and “How” type/format questions.

As another specific illustrative example, in one embodiment, an askinguser's question data is analyzed at BEFORE PROVIDING THE QUESTION DATATO ANY OF THE ONE OR MORE SUPPORT PERSONNEL FOR THE PURPOSE OF PROVIDINGAN ANSWER TO THE QUESTION, ANALYZE THE QUESTION DATA TO DETERMINE IF THEQUESTION DATA REPRESENTS AN IMPROPERLY FORMATTED QUESTION OPERATION 319as it is being entered into the question and answer based customersupport system. If the question is determined to be an improperlyformatted question because the question is in a low quality rhetorical,or an otherwise “unanswerable”, question format, then at IF ADETERMINATION IS MADE THAT THE QUESTION DATA REPRESENTS AN IMPROPERLYFORMATTED QUESTION, TAKE ONE OR MORE CORRECTIVE ACTIONS BEFORE PROVIDINGTHE QUESTION DATA TO THE ONE OR MORE SUPPORT PERSONNEL FOR THE PURPOSEOF PROVIDING AN ANSWER TO THE QUESTION OPERATION 321 the asking user isprovided format transformation instructions that guide the user througha step-by-step process to transform the identified rhetorical, orunanswerable, improperly formatted question into a properly formattedquestion in a high quality question format, such as, for example, aclosed-ended question, capable of being answered with a simple “yes” or“no”, or a closed-ended question capable of being answered bymulti-choice, or mapping. In one embodiment, the format transformationinstructions are used to implement the step-by-step transformationprocess before the question is submitted to the question and answerbased customer support system for response, and before any resources areactually devoted to trying to answer the improperly formatted question.

As another specific illustrative example, in one embodiment, an askinguser's question data is analyzed at BEFORE PROVIDING THE QUESTION DATATO ANY OF THE ONE OR MORE SUPPORT PERSONNEL FOR THE PURPOSE OF PROVIDINGAN ANSWER TO THE QUESTION, ANALYZE THE QUESTION DATA TO DETERMINE IF THEQUESTION DATA REPRESENTS AN IMPROPERLY FORMATTED QUESTION OPERATION 319as it is being entered into the question and answer based customersupport system. If the question is determined to be an improperlyformatted question because the question or query is determined to be ina low quality grammatically incorrect question or search query format,then at IF A DETERMINATION IS MADE THAT THE QUESTION DATA REPRESENTS ANIMPROPERLY FORMATTED QUESTION, TAKE ONE OR MORE CORRECTIVE ACTIONSBEFORE PROVIDING THE QUESTION DATA TO THE ONE OR MORE SUPPORT PERSONNELFOR THE PURPOSE OF PROVIDING AN ANSWER TO THE QUESTION OPERATION 321 theasking or searching user is provided format transformation instructionsthat guide the user through a step-by-step process to transform theimproperly formatted question into a properly formatted question havinga high quality grammatically correct format. In one embodiment, this thestep-by-step transformation process is implemented using the formattransformation instructions before the question or query is evensubmitted for response, and again before any resources are devoted totrying to answer the improperly formatted question.

In various embodiments, the format transformation instructions areprovided to the user through one or more question asking experiencequestion transformation interface screens used to direct users towardstransforming improperly formatted questions into properly formattedclosed-ended questions. Three illustrative examples of question-askingexperience question transformation interface screens are shown in FIGS.2A, 2B, and 2C and discussed separately below.

Referring to FIG. 2A, a first experience prototype, and associatedquestion transformation interface screen, is shown. The experienceprototype, and associated question transformation interface screen, ofFIG. 2A uses constraints, in the form of format transformationinstructions, to direct users towards asking closed-ended questions,and, in one embodiment, goes as far as defaulting to Yes/No answertypes. The experience prototype, and associated question transformationinterface screen, of FIG. 2A emphasizes closed-ended questions, whichyield the highest rated answers (see FIG. 1A).

The approach represented by the experience prototype, and associatedquestion transformation interface screen, of FIG. 2A requiresfront-loaded cognitive overhead in that it forces users to think abouttheir question type/format before they have a chance to phrase it. Ithas been empirically determined that survey-style answer choices helpphrasing. For open-ended questions, the user is prompted to transformthe question into the most highly rated question-types in order ofeffectiveness (see FIG. 1A), i.e.: “Where” type/format questions, “What”type/format questions, “When” type/format questions, “Who” type/formatquestions, and “How” type/format questions.

Referring to FIG. 2B, a second experience prototype, and associatedquestion transformation interface screen, is shown. The secondexperience prototype, and associated question transformation interfacescreen, of FIG. 2B is referred to as the question optimizer approach.The question optimizer experience prototype, and associated questiontransformation interface screen, of FIG. 2B provides formattransformation instructions that allow the user to formulate a questionfirst, then advises the user to re-phrase/transform an improperlyformatted question using personalized transformation instructions/tipsto guide the user into formulating a better question. The questionoptimizer experience prototype, and associated question transformationinterface screen, of FIG. 2B thus provides contextual cues as to how tore-phrase the question to achieve highest rated answers using the dataof FIG. 1A. Using one embodiment of the Question optimizer” experienceprototype, and associated question transformation interface screen, ofFIG. 2B users are asked to retype their question rather than edit thequestion. Research confirmed that this approach helps the usersre-phrase most effectively.

For open-ended questions, the user is prompted to transform the questioninto the most highly rated question-types in order of effectiveness (seeFIG. 1A), i.e.: “Where” type/format questions, “What” type/formatquestions, “When” type/format questions, “Who” type/format questions,and “How” type/format questions.

One advantage of the question optimizer experience prototype, andassociated question transformation interface screen, approach of FIG. 2Bis that initial text inputs from the user can be analyzed pro-activelyin real time. In most cases, the question type/format can be reliablydetected from the first few characters/tokens of the question, as thequestion is being entered. The intervention accomplished through thequestion optimizer experience prototype, and associated questiontransformation interface screen, of FIG. 2B may therefore come at thevery early stages of question formulation, alleviating user's concernabout accidentally losing the question before it is recorded/submittedto the question and answer based customer support system customersupport question and answer database.

The overall success rate of the question optimizer experience prototype,and associated question transformation interface screen, of FIG. 2B,i.e., the ability of the user to transform a “Why” type/format questionto another, preferred, question type/format was 80%. What was especiallyencouraging is that some users were able to re-phrase “Why” type/formatquestions into closed-ended category questions while keeping theoriginal intent of the question intact. This allowed us to accomplishgoal of transforming “Why” and “How” type/formatted questions into toclosed-ended category/formatted questions.

In addition, in most cases, the questions transformed/re-phrased usingthe question optimizer experience prototype, and associated questiontransformation interface screen, of FIG. 2B, became easier to understandand “answerable.” This allowed us to accomplish the goal of transforming“unanswerable” questions into “answerable” questions.

In addition, re-phrasing/transforming using the question optimizerexperience prototype, and associated question transformation interfacescreen, of FIG. 2B, typically resulted in better formed/formattedquestions compared to the original question, including proper spelling,grammar structure, capitalization and punctuation. This allowed us toaccomplish the goal of transforming ill-formed questions into “wellformed” questions.

In another experiment, we also found that users who were asked tore-type the question using the question optimizer experience prototype,and associated question transformation interface screen, of FIG. 2B,generally did a better job in re-phrasing the original question. On thecontrary, users who were asked to edit the original question typicallykept the original question type intact.

Referring to FIG. 2C, a third experience prototype, and associatedquestion transformation interface screen, is shown. The experienceprototype, and associated question transformation interface screen,shown in FIG. 2C abandons the need to choose between open or closedquestions upfront. This allows the user to submit/document theirquestion, and then with that task complete, move on tooptimizing/transforming it. The experience prototype, and associatedquestion transformation interface screen, of FIG. 2C also introduces theconcept of a visual question-quality meter 201, which provides a simplerread of question-quality. It is believed users will be more interestedin-rephrasing/transforming their question multiple times in a quest toget the meter “into the green.”

In one embodiment, once one or more corrective actions are taken if theanalysis of BEFORE PROVIDING THE QUESTION DATA TO ANY OF THE ONE OR MORESUPPORT PERSONNEL FOR THE PURPOSE OF PROVIDING AN ANSWER TO THEQUESTION, ANALYZE THE QUESTION DATA TO DETERMINE IF THE QUESTION DATAREPRESENTS AN IMPROPERLY FORMATTED QUESTION OPERATION 319 determinesthat a question represents an improperly formatted question at IF ADETERMINATION IS MADE THAT THE QUESTION DATA REPRESENTS AN IMPROPERLYFORMATTED QUESTION, TAKE ONE OR MORE CORRECTIVE ACTIONS BEFORE PROVIDINGTHE QUESTION DATA TO THE ONE OR MORE SUPPORT PERSONNEL FOR THE PURPOSEOF PROVIDING AN ANSWER TO THE QUESTION OPERATION 321, process flowproceeds to EXIT OPERATION 330.

In one embodiment, at EXIT OPERATION 330 process 300 for pro-activedetection and correction of low quality questions submitted to aquestion and answer based customer support system is exited to await newdata.

FIGS. 4A, 4B, and 4C together are a block diagram depicting a questionanalysis and reformation process for using format transformationinstructions to transform an improperly formatted question into aproperly formatted question in accordance with one embodiment.

In one embodiment, once a software system, a question and answer basedcustomer support system, and a customer support question and answerdatabase are provided as discussed above with respect to FIG. 3 andprocess 300 for pro-active detection and correction of low qualityquestions submitted to a question and answer based customer supportsystem, searching users of the software system are provided thecapability search the customer support question and answer database tofind question and answer data related to a topic or question of interestto the searching user.

Referring to FIG. 4A, at SEARCH BLOCK 403 a searching user submitssearch query data to search the customer support question and answerdatabase to find question and answer data related to the searching userstopic or question of interest. In one embodiment, results datarepresenting the results of the search query are provided to thesearching user at REVIEW RESULTS BLOCK 404.

In one embodiment, at REVIEW RESULTS BLOCK 404 results data representingthe results of the searching user's search of SEARCH BLOCK 403 arepresented to the searching user and a determination is made at FOUND?BLOCK 405 as to whether the search results address the searching user'stopic and/or answer the searching user's question.

In one embodiment, if at FOUND? BLOCK 405 a determination is made thatthe results of REVIEW RESULTS BLOCK 404 do address the searching user'stopic and/or answer the searching user's question, then process flowmoves directly to DONE/EXIT BLOCK 490.

In one embodiment, if at FOUND? BLOCK 405 a determination is made thatthe results of REVIEW RESULTS BLOCK 404 do not address the searchinguser's topic and/or answer the searching user's question, then processflow proceeds to SELECT POST A NEW QUESTION BLOCK 406.

In one embodiment, at SELECT POST A NEW QUESTION BLOCK 406 the searchinguser initiates the question submission process and thereby becomes anasking user. In one embodiment, at SELECT POST A NEW QUESTION BLOCK 406the now asking user is provided a question data entry interface screenthrough which the asking user can enter or provide question data atPROVIDE QUESTION DATA BLOCK 407.

In one embodiment, at PROVIDE QUESTION DATA BLOCK 407 the asking userbegins entering question data representing a question being submitted onbehalf of the asking user.

In one embodiment, once the question data is submitted by the askinguser at PROVIDE QUESTION DATA BLOCK 407 process flow proceeds through TO411 OF FIG. 4B BLOCK 408 of FIG. 4A to FROM 408 OF FIG. 4A BLOCK 411 ofFIG. 4B where an initial check of the grammatical format of the questionbeing submitted via the question data of PROVIDE QUESTION DATA BLOCK 407is performed.

Referring to FIG. 4B, at ENDS WITH?? BLOCK 412 a determination is madeas to whether the question represented by the question data of PROVIDEQUESTION DATA BLOCK 407 concludes a “?” symbol. If the questionrepresented by the question data of PROVIDE QUESTION DATA BLOCK 407includes a “?” symbol the question data is auto-corrected atAUTO-CORRECT? BLOCK 413.

Process flow then proceeds to SENTENCE CASE? BLOCK 414 and, if required,the sentence case of the question represented by the question data ofPROVIDE QUESTION DATA BLOCK 407 is auto corrected at AUTO-CORRECT CASEBLOCK 415.

Process flow then proceeds to MISSPELLING? BLOCK 417 where the questionrepresented by the question data of PROVIDE QUESTION DATA BLOCK 407 iscompared with SPELLING CORRECTIONS DATABASE 416 and any known misspelledwords are corrected at AUTO-CORRECT SPELLING BLOCK 418.

Process flow then proceeds to UNKNOWN WORD? BLOCK 420 where the questionrepresented by the question data of PROVIDE QUESTION DATA BLOCK 407 iscompared with DICTIONARY DATABASE 419 and any unknown words areidentified, corrected, or flagged for correction, at IDENTIFY/FLAG FORCORRECTION BLOCK 421.

Once the initial grammar analysis is performed as described above andshown in FIG. 4B, process flow proceeds from TO 431 OF FIG. 4A BLOCK 422of FIG. 4B to FROM 422 OF FIG. 4B BLOCK 431 of FIG. 4A and trough toQUESTION FORMAT ANALYSIS BLOCK 432.

As discussed in more detail above, questions submitted to the questionand answer based customer support system at PROVIDE QUESTION DATA BLOCK407 can be structured/formatted in a variety of ways and these variousquestion type/formats can vary significantly in quality, and moreimportantly, in the predicted user satisfaction with an answer, anyanswer, to the question.

As a specific illustrative example, questions submitted to the questionand answer based customer support system at PROVIDE QUESTION DATA BLOCK407 can be, but are not limited to: general knowledge/open-ended typequestions, defined as “Who” type/format questions, “What” type/formatquestions, “When” type/format questions, “Where” type/format questions,“Why” type/format questions, and “How” type/format questions;rhetorical, or otherwise “unanswerable” questions; grammaticallyincorrect questions and/or queries; otherwise ill-formed questions;and/or closed-ended questions, capable of being answered with a simple“yes” or “no”, or via a multi-choice, or mapping.

As discussed above, each of these question structures is associated witha empirically calculated predictability that the answer to the question,whatever that answer may be, will be found satisfactory by the users,e.g., the asking user and/or searching users. As also discussed above,this discovery by the inventors is leveraged to predict usersatisfaction with answers that may eventually be provided through thequestion and answer based customer support system by performingpre-submission analysis of the type/format, and other attributes, of thequestion itself, rather than post question submission analysis of ananswer provided in response to the question. This paradigm shiftingapproach to predicting user satisfaction with an answer based on theuser's question alone, and before the answer is generated, is in directcontrast to prior art assumptions and approaches that focused on theanswers provided through a question and answer based customer supportsystem, and analysis performed after both the question and answer hadalready been formulated and provided to users.

In one embodiment, the analysis performed at QUESTION FORMAT ANALYSISBLOCK 432, PROPERLY FORMATTED QUESTION? BLOCK 433, and via the processblocks of FIG. 4C discussed below, is based on an empiric questiontaxonomy taking into account one or more of, the question type/format,the question grammar structure, the type of anticipated answer, askertype, and various other factors, as discussed above with respect to FIG.3.

In one embodiment, low quality question formats that are predicted toresult in answers that will have user satisfaction ratings below athreshold level of user satisfaction are defined/identified at QUESTIONFORMAT ANALYSIS BLOCK 432 and PROPERLY FORMATTED QUESTION? BLOCK 433based on the empirical data obtained as discussed above and shown inFIG. 1A.

In one embodiment, process flow proceeds from QUESTION FORMAT ANALYSISBLOCK 432 to PROPERLY FORMATTED QUESTION? BLOCK 433. In one environment,at PROPERLY FORMATTED QUESTION? BLOCK 433, a determination is made as towhether the question represented by the question data of PROVIDEQUESTION DATA BLOCK 407 is in the proper question format. In oneembodiment, if the question represented by the question data of PROVIDEQUESTION DATA BLOCK 407 is determined to be in the proper questionformat at PROPERLY FORMATTED QUESTION? BLOCK 433, process flow proceedsto ADD DETAILS BLOCK 464.

In one embodiment, at ADD DETAILS BLOCK 464, if additional detailsregarding the question represented by the question data of PROVIDEQUESTION DATA BLOCK 407 are required, those details are requested, andprovided. In one embodiment, once any required additional detailsregarding the question represented by the question data of PROVIDEQUESTION DATA BLOCK 407 are provided, process flow proceeds to POST NEWQUESTION BLOCK 465.

In one embodiment, at POST NEW QUESTION BLOCK 465 the questionrepresented by the question data of PROVIDE QUESTION DATA BLOCK 407, andany required additional details, are posted in the customer supportquestion-and-answer database and provided to one or more customersupport personnel to be addressed and answered by one of the one or morecustomer support personnel. In one embodiment, once the questionrepresented by the question data of PROVIDE QUESTION DATA BLOCK 407, andany required additional details, are posted in the customer supportquestion-and-answer database, process flow proceeds to DONE/EXIT BLOCK490.

On the other hand, in one embodiment if at PROPERLY FORMATTED QUESTION?BLOCK 433, a determination is made that the question represented by thequestion data of PROVIDE QUESTION DATA BLOCK 407 is not in the properquestion format process flow proceeds to, TO 441 OF FIG. 4C BLOCK 434.In one embodiment, process flow then proceeds thorough TO 441 OF FIG. 4CBLOCK 434 of FIG. 4A to FIG. 4C and FROM 434 OF FIG. 4A BLOCK 441. Inone embodiment, at FROM 434 OF FIG. 4A BLOCK 441 process flow proceedsto WHY FORMAT? BLOCK 442 and question formatting analysis is performedon the question data of PROVIDE QUESTION DATA BLOCK 407.

In one embodiment, at WHY FORMAT? BLOCK 442 a determination is made asto whether the question represented by the question data of PROVIDEQUESTION DATA BLOCK 407 is in the open-end general information “Why”format. As seen in FIG. 1A, “Why” type/format questions have arelatively low satisfaction rating of 56.3%. Consequently, in variousembodiments, “Why” type/format questions would be defined at QUESTIONFORMAT ANALYSIS BLOCK 432 and PROPERLY FORMATTED QUESTION? BLOCK 433 aslow quality question formats. In one embodiment, any question formatother than the closed-ended type/format questions is defined as a lowquality question format at QUESTION FORMAT ANALYSIS BLOCK 432 andPROPERLY FORMATTED QUESTION? BLOCK 433.

In one embodiment, if at WHY FORMAT? BLOCK 442 a determination is madethat the question represented by the question data of PROVIDE QUESTIONDATA BLOCK 407 is in the open-end general information “Why” format, thenformat transformation instructions are provided at PROVIDE FORMATTRANSFORMATION INSTRUCTIONS TO ELIMINATE WHY FORMAT BLOCK 443 to guidethe user through a process of reforming the question represented by thequestion data of PROVIDE QUESTION DATA BLOCK 407 into a non-“why formatquestion.

In various embodiments, the format transformation instructions areprovided to the user through one or more question-asking experiencequestion transformation interface screens such as those shown in FIGS.2A, 2B, and 2C and discussed separately above.

In one embodiment, since, as seen in FIG. 1A, the “Why” format questionshave the lowest user satisfaction ratings, virtually any question formatis preferred over the “Why” question format. Consequently, in oneembodiment, the format transformation instructions question-askingexperience question transformation interface screens employed at PROVIDEFORMAT TRANSFORMATION INSTRUCTIONS TO ELIMINATE WHY FORMAT BLOCK 443 areused to help the user transform the “Why” format question into any ofthe other question formats with the preferred order being, closed-ended,“Who” general knowledge/open-ended, “What” general knowledge/open-ended,“Where” general knowledge/open-ended, and “How” generalknowledge/open-ended.

In one embodiment, once the “Why” formatted question is transformed intoanother question format at PROVIDE FORMAT TRANSFORMATION INSTRUCTIONS TOELIMINATE WHY FORMAT BLOCK 443, process flow proceeds to TO 461 OF FIG.4A BLOCK 451.

In one embodiment, if at WHY FORMAT? BLOCK 442 a determination is madethat the question represented by the question data of PROVIDE QUESTIONDATA BLOCK 407 is not in the open-end general information “Why” format,process flow proceeds to IMPROPER LENGTH? BLOCK 444.

In one embodiment, at IMPROPER LENGTH? BLOCK 444 a determination is madeas to whether the question represented by the question data of PROVIDEQUESTION DATA BLOCK 407 is of the required length. In one embodiment, ifat IMPROPER LENGTH? BLOCK 444 a determination is made that the questionrepresented by the question data of PROVIDE QUESTION DATA BLOCK 407 isnot of the required length, process flow proceeds to PROVIDE FORMATTRANSFORMATION INSTRUCTIONS TO RE-WRITE TO PROPER LENGTH BLOCK 445. Inone embodiment, at PROVIDE FORMAT TRANSFORMATION INSTRUCTIONS TORE-WRITE TO PROPER LENGTH BLOCK 445 format transformation instructionsare provided to guide the user through a process of reforming thequestion represented by the question data of PROVIDE QUESTION DATA BLOCK407 into a question of the proper length.

In various embodiments, the format transformation instructions areprovided to the user through one or more question-asking experiencequestion transformation interface screens such as those shown in FIGS.2A, 2B, and 2C and discussed separately above.

In one embodiment, once the question represented by the question data ofPROVIDE QUESTION DATA BLOCK 407 is transformed into a question of theproper length at PROVIDE FORMAT TRANSFORMATION INSTRUCTIONS TO RE-WRITETO PROPER LENGTH BLOCK 445, process flow proceeds to TO 461 OF FIG. 4ABLOCK 451.

In one embodiment, if at IMPROPER LENGTH? BLOCK 444 a determination ismade that the question represented by the question data of PROVIDEQUESTION DATA BLOCK 407 is of the proper length, process flow proceedsto CLOSED ENDED FORMAT? BLOCK 446.

In one embodiment, at CLOSED ENDED FORMAT? BLOCK 446 a determination ismade as to whether the question represented by the question data ofPROVIDE QUESTION DATA BLOCK 407 is in a closed-ended format. In oneembodiment, if at CLOSED ENDED FORMAT? BLOCK 446 a determination is madethat the question represented by the question data of PROVIDE QUESTIONDATA BLOCK 407 is not in a closed-ended format, process flow proceeds toPROVIDE FORMAT TRANSFORMATION INSTRUCTIONS TO TRANSFORM TO CLOSED-ENDEDFORMAT BLOCK 447.

In one embodiment, at PROVIDE FORMAT TRANSFORMATION INSTRUCTIONS TOTRANSFORM TO IDEAL CLOSED-ENDED FORMAT BLOCK 447 format transformationinstructions are provided to guide the user through a process ofreforming the question represented by the question data of PROVIDEQUESTION DATA BLOCK 407 into a question in the preferred closed-endedformat.

As noted above, closed-ended type/format questions have a very highsatisfaction rating of 85.9%. Consequently, in various embodiments, allquestions of the closed-ended type/format are defined as properlyformatted questions and any question in a question format other than the“closed ended” format is defined as an improperly formatted question.

As noted above, most closed-ended category format questions are in thesub-category of “Yes-No” type questions. These “Yes-No” type questionsare identified by the fact that they typically start with an auxiliaryverb such as “Do”, “Can”, “Be.” As also noted above, the secondsub-category of closed-ended question format includes “Choice” typequestions. These “Choice” type questions are identified by the fact thatthey generally start with an auxiliary verb and also contain theconjunction “or.” In one embodiment, the “Yes-No” type closed-endedformat questions are the most preferred closed-ended format.

In various embodiments, the format transformation instructions areprovided to the user through one or more question-asking experiencequestion transformation interface screens such as those shown in FIGS.2A, 2B, and 2C and discussed separately above.

In one embodiment, once the question represented by the question data ofPROVIDE QUESTION DATA BLOCK 407 is transformed into a closed-endedformatted question at PROVIDE FORMAT TRANSFORMATION INSTRUCTIONS TOTRANSFORM TO CLOSED-ENDED FORMAT BLOCK 447, process flow proceeds to TO461 OF FIG. 4A BLOCK 451.

In one embodiment, if at CLOSED ENDED FORMAT? BLOCK 446 a determinationcannot be made that the question represented by the question data ofPROVIDE QUESTION DATA BLOCK 407 is in the closed-end question format,process flow proceeds to OPEN ENDED FORMAT? BLOCK 448.

In one embodiment, at OPEN ENDED FORMAT? BLOCK 448, a determination ismade as to whether the question represented by the question data ofPROVIDE QUESTION DATA BLOCK 407 is in an open-ended format. In oneembodiment, if at OPEN ENDED FORMAT? BLOCK 448 a determination is madethat the question represented by the question data of PROVIDE QUESTIONDATA BLOCK 407 is in an open-ended format, process flow proceeds toPROVIDE FORMAT TRANSFORMATION INSTRUCTIONS TO TRANSFORM TO CLOSED-ENDEDOR BEST OPEN-ENDED FORMAT BLOCK 449.

In one embodiment, at PROVIDE FORMAT TRANSFORMATION INSTRUCTIONS TOTRANSFORM TO CLOSED-ENDED OR BEST OPEN-ENDED FORMAT BLOCK 449 formattransformation instructions are provided to guide the user through aprocess of reforming the question represented by the question data ofPROVIDE QUESTION DATA BLOCK 407 into a question in the preferredclosed-ended format, or at least a more preferred open-ended questionformat.

As seen in FIG. 1A, the general knowledge/open-ended category questionshave different levels of user satisfaction ratings. However, none of theknowledge/open-ended category questions have user satisfaction ratingsas high as the 85.9% user satisfaction rating of the closed-endedquestion format. Consequently, all questions of the closed-endedtype/format are defined as properly formatted questions and any questionin a question format other than the “closed ended” format is defined asan improperly formatted question. As a result, if at OPEN ENDED FORMAT?BLOCK 448 a determination is made that the question represented by thequestion data of PROVIDE QUESTION DATA BLOCK 407 is in an open-endedformat, format transformation instructions are provided at PROVIDEFORMAT TRANSFORMATION INSTRUCTIONS TO TRANSFORM TO CLOSED-ENDED OR BESTOPEN-ENDED FORMAT BLOCK 449 to attempt to guide the user through aprocess of reforming the question represented by the question data ofPROVIDE QUESTION DATA BLOCK 407 into a question in the preferredclosed-ended format. As noted above, the “Yes-No” type closed-endedformat questions are the most preferred closed-ended format.

However, in some cases, the goal of reforming the question representedby the question data of PROVIDE QUESTION DATA BLOCK 407 into a questionin the preferred closed-ended format is not attainable. In theseinstances, if at OPEN ENDED FORMAT? BLOCK 448 a determination is madethat the question represented by the question data of PROVIDE QUESTIONDATA BLOCK 407 is in an open-ended format, format transformationinstructions are provided at PROVIDE FORMAT TRANSFORMATION INSTRUCTIONSTO TRANSFORM TO CLOSED-ENDED OR BEST OPEN-ENDED FORMAT BLOCK 449 toattempt to guide the user through a process of reforming the questionrepresented by the question data of PROVIDE QUESTION DATA BLOCK 407 intoa question in the best open-ended question format that can be attained.

To this end, in one embodiment, general knowledge/open-ended categoryquestions submitted are identified. As noted above, generalknowledge/open-ended category questions are of the form “Who,” “What,”“Where,” “When,” “How,” and “Why” formatted questions. Consequently, inone embodiment, the question data is analyzed to detect these terms, ortheir functional equivalents.

In one embodiment, then format transformation instructions are providedat PROVIDE FORMAT TRANSFORMATION INSTRUCTIONS TO TRANSFORM TOCLOSED-ENDED OR BEST OPEN-ENDED FORMAT BLOCK 449 to attempt to guide theuser through a process of reforming the question represented by thequestion data of PROVIDE QUESTION DATA BLOCK 407 into a question in thebest open-ended question format that can be attained with the preferredorder being, closed-ended, “Who” general knowledge/open-ended, “What”general knowledge/open-ended, “Where” general knowledge/open-ended, and“How” general knowledge/open-ended.

In various embodiments, the format transformation instructions areprovided to the user through one or more question-asking experiencequestion transformation interface screens such as those shown in FIGS.2A, 2B, and 2C and discussed separately above.

In one embodiment, once the question represented by the question data ofPROVIDE QUESTION DATA BLOCK 407 is transformed into a closed-endedformatted question, or at least a more preferred open-ended questionformat question, at PROVIDE FORMAT TRANSFORMATION INSTRUCTIONS TOTRANSFORM TO CLOSED-ENDED OR BEST OPEN-ENDED FORMAT BLOCK 449, processflow proceeds to TO 461 OF FIG. 4A BLOCK 451.

In one embodiment, once the question format analysis of FIG. 4C, and WHYFORMAT? BLOCK 442, IMPROPER LENGTH? BLOCK 444, CLOSED ENDED FORMAT?BLOCK 446, and OPEN ENDED FORMAT? BLOCK 448 is complete, process flowproceeds through TO 461 OF FIG. 4A BLOCK 451 of FIG. 4C to FROM 451 OFFIG. 4C BLOCK 461 of FIG. 4A, and RE-WRITE QUESTION BLOCK 462.

In on embodiment, at RE-WRITE QUESTION BLOCK 462 the questionrepresented by the question data of PROVIDE QUESTION DATA BLOCK 407 isrewritten in accordance with the procedures and guidance of thesub-processes of FIG. 4B and FIG. 4C discussed above. In one embodiment,the rewriting of the question represented by the question data ofPROVIDE QUESTION DATA BLOCK 407 is analyzed at RE-WRITE QUESTION BLOCK462 and if a threshold change in the question represented by thequestion data of PROVIDE QUESTION DATA BLOCK 407 is detected atTHRESHOLD CHANGE? BLOCK 463, then the question represented by thequestion data of PROVIDE QUESTION DATA BLOCK 407 is determined to be ade facto new question. Consequently, a second search of the customersupport question-and-answer database is conducted at NEW SEARCH BLOCK471 using the rewritten question data.

In one embodiment, if, as a result of the new search of the customersupport question-and-answer database using the using the rewrittenquestion data at NEW SEARCH BLOCK 471 results data addressing the topicof the using the rewritten question data are received FOUND? BLOCK 472,then process flow proceeds to DONE/EXIT BLOCK 490.

In one embodiment, if, as a result of the new search of the customersupport question-and-answer database using the using the rewrittenquestion data at NEW SEARCH BLOCK 471 results data addressing the topicof the using the rewritten question data are not received FOUND? BLOCK472, then process flow proceeds to ADD DETAILS BLOCK 464.

In one embodiment, if a threshold change in the question represented bythe question data of PROVIDE QUESTION DATA BLOCK 407 is not detected,then process flow proceeds to ADD DETAILS BLOCK 464. In one embodiment,at ADD DETAILS BLOCK 464, if additional details regarding the questionrepresented by the question data of PROVIDE QUESTION DATA BLOCK 407 arerequired, those details are requested, and provided. In one embodiment,once any required additional details regarding the question representedby the question data of PROVIDE QUESTION DATA BLOCK 407 are provided,process flow proceeds to POST NEW QUESTION BLOCK 465.

In one embodiment, at POST NEW QUESTION BLOCK 465 the questionrepresented by the question data of PROVIDE QUESTION DATA BLOCK 407, andany required additional details, are posted in the customer supportquestion-and-answer database and provided to one or more customersupport personnel to be addressed and answered by one of the one or morecustomer support personnel. In one embodiment, once the questionrepresented by the question data of PROVIDE QUESTION DATA BLOCK 407, andany required additional details, are posted in the customer supportquestion-and-answer database, process flow proceeds to DONE/EXIT BLOCK490.

FIG. 5 is a block diagram of a hardware and production environmentsystem 500 for providing a process for pro-active detection andcorrection of low quality questions submitted to a question and answerbased customer support system in accordance with one embodiment.

As seen in FIG. 5, in one embodiment, a provider computing system 503 isprovided in provider computing environment 501 and includes softwaresystem 505. In various embodiments, software system 505 is any softwaresystem discussed herein, known at the time of filing, and/or asdeveloped after the time of filing.

As also seen in FIG. 5, user computing system 523 is provided in usercomputing environment 521. In one embodiment, a user of software system505 accesses provider computing system 503 and software system 505 viacommunications channel 571.

In one embodiment, the users of software system 505 are also provided aquestion and answer based customer support system 535 shown asimplemented on question and answer based customer support systemcomputing system 533 in question and answer based customer supportsystem computing environment 531.

In one embodiment, through question and answer based customer supportsystem 535, users can submit question data 525 via communicationschannel 573. In one embodiment, question data 525 entered by the usersrepresents questions to be potentially be provided to one or moresupport personnel associated with question and answer based customersupport system 535. In one embodiment, question data 525 is submitted bythe users so that the questions represented by question data 525 can bepotentially be answered by at least one of the one or more supportpersonnel associated with support personnel computing system 553 shownas implemented in support personnel computing environment 551.

In one embodiment, low quality question formats that are predicted toresult in answers that will have user satisfaction ratings below athreshold level of user satisfaction are defined/identified and recordedin low quality format question data 537. In one embodiment, anyidentified questions submitted via question data 525 in any of the lowquality question formats of low quality format question data 537 aredefined/labeled as improperly formatted questions.

Alternatively, in one embodiment, high quality question formats that arepredicted to result in answers that will have user satisfaction ratingsabove a threshold level of user satisfaction are defined/identified andrecorded in high quality format question data 539. In one embodiment,any identified questions submitted via question data 525 in any of thehigh quality question formats of high quality format question data 539are defined/labeled as properly formatted questions.

In one embodiment, when question data 525 representing a questionsubmitted by a user through question and answer based customer supportsystem 535 is being entered by a user, and/or is otherwise received byquestion and answer based customer support system 535 at question datareceiving module 543 communications channel 573, question data 525 isanalyzed by question format analysis module 541 using low quality formatquestion data 537 and/or high quality format question data 539 beforeproviding question data 525 to any of the one or more support personnelat support personnel computing system 553 to answer to the questionrepresented by question data 525 to determine if question data 525represents an improperly formatted question.

In one embodiment, if, based on the analysis of question data 525 atquestion format analysis module 541, a determination is made thatquestion data 525 represents an improperly formatted question, one ormore corrective actions are implemented by corrective action module 545.

In one embodiment, the one or more corrective actions are implemented bycorrective action module 545 using one or more of, corrective action 1instructions 546A, corrective action 2 instructions 546B, correctiveaction 3 instructions 546C, through corrective action N instructions546N, before providing the question data 525 to support personnel accessmodule 547, support personnel access portal 549, and the supportpersonnel computing system 553 via communications channel 575, to answerthe question represented by question data 525.

In one embodiment, the one or more corrective actions taken beforeproviding the question data to the one or more support personnel ofcorrective action module 545 includes filtering out the improperlyformatted question before the improperly formatted question is providedto the support community, and before any resources are devoted toanswering the improperly formatted question.

In one embodiment, the one or more corrective actions taken beforeproviding the question data to the one or more support personnel ofcorrective action module 545 includes refusing to accept submission ofthe improperly formatted question before the improperly formattedquestion is provided to the support community, and before any resourcesare devoted to answering the improperly formatted question.

In one embodiment, the one or more corrective actions taken beforeproviding the question data to the one or more support personnel ofcorrective action module 545 includes attempting to correct theimproperly formatted question by providing the user with a set ofquestion transformation instructions used to transform the improperlyformatted question into a properly formatted question.

Consequently, improperly formatted questions of question data 525submitted to question and answer based customer support system 535 areidentified before the questions are submitted to support personnelcomputing system 553, and/or any support personnel, for the purpose ofproviding an answer to the question, and before any resources areexpended in an attempt to answer the improperly formatted question. Inaddition, improperly formatted questions are identified before anyusers, including the asking user, are provided answers to improperlyformatted questions that are likely to result in low user satisfactionratings.

Using processes 300 and/or 400, and system 500, as disclosed herein,satisfaction with answers to questions that may eventually be providedthrough a question and answer based customer support system can bepredicted before the questions are formally submitted to the questionand answer based customer support system and/or channeled to the supportcommunity for analysis and answering. Therefore, the concepts disclosedherein provide an opportunity to intervene in the question draftingprocess, in relative real time, while the question is still beingformulated, and before any resources are devoted to actually trying toanswer improperly formatted, i.e., low quality, questions. Consequently,in one embodiment, the user is coached during the user's questionformulation, i.e., during user's entry of the question data representingthe question, in such a way that there is a significantly higherlikelihood that not only the asking user will be satisfied with theanswer eventually provided, but that other searching users accessing thequestion and answer pair through a question and answer database willalso be satisfied with the answer eventually provided. Therefore,processes 300 and/or 400, and system 500, as disclosed herein alsoprovides for significant improvements to the technical fields ofcustomer support, information dissemination, software implementation,and user experience.

In addition, using processes 300 and/or 400, and system 500, results inmore efficient use of human and non-human resources, fewer processorcycles being utilized, reduced memory utilization, and lesscommunications bandwidth being utilized to relay data to and frombackend systems. As a result, computing systems are transformed intofaster, more efficient, and more effective computing systems byimplementing processes 300 and/or 400, and system 500, as disclosedherein.

The various embodiments of the disclosure can be implemented to improvethe technical fields of customer support, information dissemination,software implementation, and user experience. Therefore, the variousdescribed embodiments of the disclosure and their associated benefitsamount to significantly more than an abstract idea.

The present invention has been described in particular detail withrespect to specific possible embodiments. Those of skill in the art willappreciate that the invention may be practiced in other embodiments. Forexample, the nomenclature used for components, capitalization ofcomponent designations and terms, the attributes, data structures, orany other programming or structural aspect is not significant,mandatory, or limiting, and the mechanisms that implement the inventionor its features can have various different names, formats, and/orprotocols. Further, the system and/or functionality of the invention maybe implemented via various combinations of software and hardware, asdescribed, or entirely in hardware elements. Also, particular divisionsof functionality between the various components described herein, aremerely exemplary, and not mandatory or significant. Consequently,functions performed by a single component may, in other embodiments, beperformed by multiple components, and functions performed by multiplecomponents may, in other embodiments, be performed by a singlecomponent.

Some portions of the above description present the features of thepresent invention in terms of algorithms and symbolic representations ofoperations, or algorithm-like representations, of operations oninformation/data. These algorithmic and/or algorithm-like descriptionsand representations are the means used by those of skill in the art tomost effectively and efficiently convey the substance of their work toothers of skill in the art. These operations, while describedfunctionally or logically, are understood to be implemented by computerprograms and/or computing systems. Furthermore, it has also provenconvenient at times to refer to these arrangements of operations assteps or modules or by functional names, without loss of generality.

Unless specifically stated otherwise, as would be apparent from theabove discussion, it is appreciated that throughout the abovedescription, discussions utilizing terms such as “accessing,”“analyzing,” “obtaining,” “identifying,” “associating,” “aggregating,”“initiating,” “collecting,” “creating,” “transferring,” “storing,”“searching,” “comparing,” “providing,” “processing” etc., refer to theaction and processes of a computing system or similar electronic devicethat manipulates and operates on data represented as physical(electronic) quantities within the computing system memories, resisters,caches or other information storage, transmission or display devices.

Certain aspects of the present invention include process steps oroperations and instructions described herein in an algorithmic and/oralgorithmic-like form. It should be noted that the process steps and/oroperations and instructions of the present invention can be embodied insoftware, firmware, and/or hardware, and when embodied in software, canbe downloaded to reside on and be operated from different platforms usedby real time network operating systems.

The present invention also relates to an apparatus or system forperforming the operations described herein. This apparatus or system maybe specifically constructed for the required purposes by a computerprogram stored via a computer program product as defined herein that canbe accessed by a computing system or other device to transform thecomputing system or other device into a specifically and speciallyprogrammed computing system or other device.

Those of skill in the art will readily recognize that the algorithms andoperations presented herein are not inherently related to any particularcomputing system, computer architecture, computer or industry standard,or any other specific apparatus. It may prove convenient/efficient toconstruct or transform one or more specialized apparatuses to performthe required operations described herein. The required structure for avariety of these systems will be apparent to those of skill in the art,along with equivalent variations. In addition, the present invention isnot described with reference to any particular programming language andit is appreciated that a variety of programming languages may be used toimplement the teachings of the present invention as described herein,and any references to a specific language or languages are provided forillustrative purposes only and for enablement of the contemplated bestmode of the invention at the time of filing.

The present invention is well suited to a wide variety of computernetwork systems operating over numerous topologies. Within this field,the configuration and management of large networks comprise storagedevices and computers that are communicatively coupled to similar and/ordissimilar computers and storage devices over a private network, a LAN,a WAN, a private network, or a public network, such as the Internet.

It should also be noted that the language used in the specification hasbeen principally selected for readability, clarity, and instructionalpurposes, and may not have been selected to delineate or circumscribethe inventive subject matter. Accordingly, the disclosure of the presentinvention is intended to be illustrative, but not limiting, of the scopeof the invention, which is set forth in the claims below.

In addition, the operations shown in the FIG.s are identified using aparticular nomenclature for ease of description and understanding, butother nomenclature is often used in the art to identify equivalentoperations.

In the discussion above, certain aspects of one embodiment includeprocess steps and/or operations and/or instructions described herein forillustrative purposes in a particular order and/or grouping. However,the particular order and/or grouping shown and discussed herein isillustrative only and not limiting. Those of skill in the art willrecognize that other orders and/or grouping of the process steps and/oroperations and/or instructions are possible and, in some embodiments,one or more of the process steps and/or operations and/or instructionsdiscussed above can be combined and/or deleted. In addition, portions ofone or more of the process steps and/or operations and/or instructionscan be re-grouped as portions of one or more other of the process stepsand/or operations and/or instructions discussed herein. Consequently,the particular order and/or grouping of the process steps and/oroperations and/or instructions discussed herein does not limit the scopeof the invention as claimed below.

Therefore, numerous variations, whether explicitly provided for by thespecification or implied by the specification or not, may be implementedby one of skill in the art in view of this disclosure.

What is claimed is:
 1. A method for pro-active detection and correctionof low quality questions submitted to a question and answer basedcustomer support system, the method comprising: providing a softwaresystem to one or more users; providing users of the software system aquestion and answer based customer support system through which questiondata can be entered by the users, the question data representingquestions to potentially be provided to one or more support personnelassociated with the question and answer based customer support system sothat the questions represented by the question data can be answered byat least one of the one or more support personnel; defining low qualityquestion formats that are predicted to result in answers that will haveuser satisfaction ratings below a threshold level of user satisfaction;defining questions having low quality question formats as improperlyformatted questions; receiving question data representing a questionsubmitted by a user through the question and answer based customersupport system; before providing the question data to any of the one ormore support personnel, analyzing the question data to determine if thequestion data represents an improperly formatted question; and if, basedon the analysis of the question data, a determination is made that thequestion data represents an improperly formatted question, taking one ormore corrective actions before providing the question data to the one ormore support personnel.
 2. The method for pro-active detection andcorrection of low quality questions submitted to a question and answerbased customer support system of claim 1 wherein the software system isselected from the group of software systems consisting of: a computingsystem implemented tax preparation software system; a network accessedtax preparation software system; a web-based tax preparation softwaresystem; a cloud-based tax preparation software system; a computingsystem implemented business management software system; a networkaccessed business management software system; a web-based businessmanagement software system; a cloud-based business management softwaresystem; a computing system implemented accounting software system; anetwork accessed accounting software system; a web-based accountingsoftware system; a cloud-based accounting software system; a computingsystem implemented financial management system; a network accessedfinancial management system; a web-based financial management system;and a cloud-based financial management system.
 3. The method forpro-active detection and correction of low quality questions submittedto a question and answer based customer support system of claim 1wherein the question and answer based customer support system includes aweb-based question and answer forum associated with the software systemand/or an area of endeavor of the software system.
 4. The method forpro-active detection and correction of low quality questions submittedto a question and answer based customer support system of claim 1wherein the question and answer based customer support system includes acustomer support question and answer database, the customer supportquestion and answer database including question and answer datarepresenting one or more questions submitted by asking users of thesoftware system and the answers to those questions provided by the oneor more support personnel, further wherein; the question and answer datain the customer support question and answer database can be searched andaccessed by searching users of the software system.
 5. The method forpro-active detection and correction of low quality questions submittedto a question and answer based customer support system of claim 4wherein answers provided by the one or more support personnel toquestions submitted by asking users of the software system can be ratedby both the asking user who submitted the question data and searchingusers who access the question and answer data in the customer supportquestion and answer database.
 6. The method for pro-active detection andcorrection of low quality questions submitted to a question and answerbased customer support system of claim 1 wherein the question data issubmitted by asking users of the software system through question datasubmission interfaces provided through the question and answer basedcustomer support system, the question data submission interfacesincluding one or more question data entry fields to enter the questiondata, the question data to be analyzed before being provided to any ofthe one or more support personnel to determine if the question datarepresents an improperly formatted question.
 7. The method forpro-active detection and correction of low quality questions submittedto a question and answer based customer support system of claim 1wherein low quality question formats that are predicted to result inanswers that will have user satisfaction ratings below a threshold levelof user satisfaction include: “Who” question formats; “What” questionformats; “When” question formats; “Where” question formats; “Why”question formats; “How” question formats; rhetorical question formats;grammatically incorrect question formats; and ill-formed questionformats.
 8. The method for pro-active detection and correction of lowquality questions submitted to a question and answer based customersupport system of claim 1 wherein if, based on the analysis of thequestion data, a determination is made that the question data representsan improperly formatted question, the one or more corrective actionstaken before providing the question data to the one or more supportpersonnel includes filtering out the improperly formatted questionbefore the improperly formatted question is provided to the supportcommunity, and before any resources are devoted to answering theimproperly formatted question.
 9. The method for pro-active detectionand correction of low quality questions submitted to a question andanswer based customer support system of claim 1 wherein if, based on theanalysis of the question data, a determination is made that the questiondata represents an improperly formatted question, the one or morecorrective actions taken before providing the question data to the oneor more support personnel includes refusing to accept submission of theimproperly formatted question before the improperly formatted questionis provided to the support community, and before any resources aredevoted to answering the improperly formatted question.
 10. The methodfor pro-active detection and correction of low quality questionssubmitted to a question and answer based customer support system ofclaim 1 wherein if, based on the analysis of the question data, adetermination is made that the question data represents an improperlyformatted question, the one or more corrective actions taken beforeproviding the question data to the one or more support personnelincludes attempting to correct the improperly formatted question byproviding the user with a set of question transformation instructionsused to transform the improperly formatted question into a properlyformatted question.
 11. The method for pro-active detection andcorrection of low quality questions submitted to a question and answerbased customer support system of claim 10 wherein the user is providedthe format transformation instructions representing suggestions on howto re-phrase/reform the improperly formatted question that arecustomized to the specific question data being submitted.
 12. The methodfor pro-active detection and correction of low quality questionssubmitted to a question and answer based customer support system ofclaim 11 wherein if, based on the analysis of the question data, adetermination is made that the question data represents an improperlyformatted question because the question represented by the question datais a general knowledge/open-ended format question, the asking user isprovided format transformation instructions that guide the user througha step-by-step process to transform the identified generalknowledge/open-ended format question into properly formattedclosed-ended question format.
 13. The method for pro-active detectionand correction of low quality questions submitted to a question andanswer based customer support system of claim 11 wherein if, based onthe analysis of the question data, a determination is made that thequestion data represents an improperly formatted question because thequestion represented by the question data is in a low qualityrhetorical, or otherwise “unanswerable”, question format, the askinguser is provided format transformation instructions that guide the userthrough a step-by-step process to transform the identified rhetorical,or otherwise “unanswerable”, format question into properly formattedclosed-ended question format.
 14. The method for pro-active detectionand correction of low quality questions submitted to a question andanswer based customer support system of claim 11 wherein if, based onthe analysis of the question data, a determination is made that thequestion data represents an improperly formatted question because thequestion represented by the question data is in a grammaticallyincorrect question or search query format, the asking user is providedformat transformation instructions that guide the user through astep-by-step process to transform the identified grammatically incorrectquestion or search query format question into properly formattedclosed-ended question format.
 15. The method for pro-active detectionand correction of low quality questions submitted to a question andanswer based customer support system of claim 14 wherein the user isprovided the format transformation instructions via a question-askingexperience question transformation interface screen.
 16. The method forpro-active detection and correction of low quality questions submittedto a question and answer based customer support system of claim 15wherein question-asking experience question transformation interfacescreen is a question optimizer question-asking experience questiontransformation interface screen.
 17. The method for pro-active detectionand correction of low quality questions submitted to a question andanswer based customer support system of claim 16 wherein the user isprovided the format transformation instructions representing suggestionson how to re-phrase/reform the improperly formatted question through thequestion optimizer question-asking experience question transformationinterface screen as the question data is being submitted.
 18. A methodfor pro-active detection and correction of low quality questionssubmitted to a question and answer based customer support system, themethod for pro-active detection and correction of low quality questionssubmitted to a question and answer based customer support systemcomprising: providing a software system to one or more users; providingusers of the software system a question and answer based customersupport system through which question data can be entered by the users,the question data representing questions to potentially be provided toone or more support personnel associated with the question and answerbased customer support system so that the questions represented by thequestion data can be answered by at least one of the one or more supportpersonnel; defining high quality question formats that are predicted toresult in answers that will have user satisfaction ratings above athreshold level of user satisfaction; defining questions having highquality question formats as properly formatted questions; receivingquestion data representing a question submitted by a user through thequestion and answer based customer support system; before providing thequestion data to any of the one or more support personnel, analyzing thequestion data to determine if the question data represents a properlyformatted question; and if, based on the analysis of the question data,a determination is made that the question data does not represent aproperly formatted question, taking one or more corrective actionsbefore providing the question data to the one or more support personnel.19. The method for pro-active detection and correction of low qualityquestions submitted to a question and answer based customer supportsystem of claim 18 wherein the software system is selected from thegroup of software systems consisting of: a computing system implementedtax preparation software system; a network accessed tax preparationsoftware system; a web-based tax preparation software system; acloud-based tax preparation software system; a computing systemimplemented business management software system; a network accessedbusiness management software system; a web-based business managementsoftware system; a cloud-based business management software system; acomputing system implemented accounting software system; a networkaccessed accounting software system; a web-based accounting softwaresystem; a cloud-based accounting software system; a computing systemimplemented financial management system; a network accessed financialmanagement system; a web-based financial management system; and acloud-based financial management system.
 20. The method for pro-activedetection and correction of low quality questions submitted to aquestion and answer based customer support system of claim 18 whereinthe question and answer based customer support system includes aweb-based question and answer forum associated with the software systemand/or an area of endeavor of the software system.
 21. The method forpro-active detection and correction of low quality questions submittedto a question and answer based customer support system of claim 18wherein the question and answer based customer support system includes acustomer support question and answer database, the customer supportquestion and answer database including question and answer datarepresenting one or more questions submitted by asking users of thesoftware system and the answers to those questions provided by the oneor more support personnel, further wherein; the question and answer datain the customer support question and answer database can be searched andaccessed by searching users of the software system.
 22. The method forpro-active detection and correction of low quality questions submittedto a question and answer based customer support system of claim 21wherein answers provided by the one or more support personnel toquestions submitted by asking users of the software system can be ratedby both the asking user who submitted the question data and searchingusers who access the question and answer data in the customer supportquestion and answer database.
 23. The method for pro-active detectionand correction of low quality questions submitted to a question andanswer based customer support system of claim 18 wherein the questiondata is submitted by asking users of the software system throughquestion data submission interfaces provided through the question andanswer based customer support system, the question data submissioninterfaces including one or more question data entry fields used toenter the question data, the question data to be analyzed before beingprovided to any of the one or more support personnel to determine if thequestion data represents a properly formatted question.
 24. The methodfor pro-active detection and correction of low quality questionssubmitted to a question and answer based customer support system ofclaim 18 wherein high quality question formats that are predicted toresult in answers that will have user satisfaction ratings above athreshold level of user satisfaction include closed-ended formattedquestions.
 25. The method for pro-active detection and correction of lowquality questions submitted to a question and answer based customersupport system of claim 18 wherein if, based on the analysis of thequestion data, a determination is made that the question data does notrepresent a properly formatted question, the one or more correctiveactions taken before providing the question data to the one or moresupport personnel includes filtering out the question before thequestion is provided to the support community, and before any resourcesare devoted to answering the question.
 26. The method for pro-activedetection and correction of low quality questions submitted to aquestion and answer based customer support system of claim 18 whereinif, based on the analysis of the question data, a determination is madethat the question data does not represent a properly formatted question,the one or more corrective actions taken before providing the questiondata to the one or more support personnel includes refusing to acceptsubmission of the question.
 27. The method for pro-active detection andcorrection of low quality questions submitted to a question and answerbased customer support system of claim 18 wherein if, based on theanalysis of the question data, a determination is made that the questiondata does not represent a properly formatted question, the one or morecorrective actions taken before providing the question data to the oneor more support personnel includes attempting to correct the question byproviding the user with a set of question transformation instructionsused to transform the question into a properly formatted question. 28.The method for pro-active detection and correction of low qualityquestions submitted to a question and answer based customer supportsystem of claim 27 wherein the user is provided the formattransformation instructions representing suggestions on how tore-phrase/reform the question that are customized to the specificquestion data being submitted.
 29. The method for pro-active detectionand correction of low quality questions submitted to a question andanswer based customer support system of claim 28 wherein if, based onthe analysis of the question data, a determination is made that thequestion data does not represent a properly formatted question becausethe question represented by the question data is a generalknowledge/open-ended format question, the asking user is provided formattransformation instructions that guide the user through a step-by-stepprocess to transform the identified general knowledge/open-ended formatquestion into a properly formatted closed-ended question format.
 30. Themethod for pro-active detection and correction of low quality questionssubmitted to a question and answer based customer support system ofclaim 28 wherein if, based on the analysis of the question data, adetermination is made that the question data does not represent aproperly formatted question because the question represented by thequestion data is in a low quality rhetorical, or an otherwise“unanswerable”, question format, then the asking user is provided formattransformation instructions that guide the user through a step-by-stepprocess to transform the identified rhetorical, or an otherwise“unanswerable” format question into a properly formatted closed-endedquestion format.
 31. The method for pro-active detection and correctionof low quality questions submitted to a question and answer basedcustomer support system of claim 30 wherein if, based on the analysis ofthe question data, a determination is made that the question data doesnot represent a properly formatted question because the questionrepresented by the question data is in a grammatically incorrectquestion or search query format, the asking user is provided formattransformation instructions that guide the user through a step-by-stepprocess to transform the identified grammatically incorrect question orsearch query format question into a properly formatted closed-endedquestion format.
 32. The method for pro-active detection and correctionof low quality questions submitted to a question and answer basedcustomer support system of claim 27 wherein the user is provided theformat transformation instructions representing suggestions on how tore-phrase/reform the improperly formatted question via a question-askingexperience question transformation interface screen.
 33. The method forpro-active detection and correction of low quality questions submittedto a question and answer based customer support system of claim 32wherein question-asking experience question transformation interfacescreen is a question optimizer question-asking experience questiontransformation interface screen.
 34. The method for pro-active detectionand correction of low quality questions submitted to a question andanswer based customer support system of claim 33 wherein the user isprovided the format transformation instructions representing suggestionson how to re-phrase/reform the improperly formatted question through thequestion optimizer question-asking experience question transformationinterface screen as the question data is being submitted.
 35. A systemfor pro-active detection and correction of low quality questionssubmitted to a question and answer based customer support system, thesystem comprising: a software system; a computing system for providingthe software system to one or more users; a question and answer basedcustomer support system through which question data can be entered bythe users, the question data representing questions to potentially beprovided to one or more support personnel associated with the questionand answer based customer support system so that the questionsrepresented by the question data can be answered by at least one of theone or more support personnel; a computing system for providing usersaccess to the question and answer based customer support system and forproviding the users the capability to enter the question datarepresenting questions to potentially be provided to one or more supportpersonnel low quality question format data defining low quality questionformats that are predicted to result in answers that will have usersatisfaction ratings below a threshold level of user satisfaction,questions having low quality question formats being defined asimproperly formatted questions; a question data receiving module forreceiving question data representing a question submitted by a userthrough the question and answer based customer support system; anquestion format analysis module, the question format analysis moduleanalyzing the question data received at the question data receivingmodule to determine if the question data represents an improperlyformatted question before providing the question data to any of the oneor more support personnel; and a corrective action module, thecorrective action module implementing one or more corrective actions ifbased on the analysis of the question data, a determination is made thatthe question data represents an improperly formatted question before thequestion data is provided to the one or more support personnel.
 36. Thesystem for pro-active detection and correction of low quality questionssubmitted to a question and answer based customer support system ofclaim 35 wherein the software system is selected from the group ofsoftware systems consisting of: a computing system implemented taxpreparation software system; a network accessed tax preparation softwaresystem; a web-based tax preparation software system; a cloud-based taxpreparation software system; a computing system implemented businessmanagement software system; a network accessed business managementsoftware system; a web-based business management software system; acloud-based business management software system; a computing systemimplemented accounting software system; a network accessed accountingsoftware system; a web-based accounting software system; a cloud-basedaccounting software system; a computing system implemented financialmanagement system; a network accessed financial management system; aweb-based financial management system; and a cloud-based financialmanagement system.
 37. The system for pro-active detection andcorrection of low quality questions submitted to a question and answerbased customer support system of claim 35 wherein the question andanswer based customer support system includes a web-based question andanswer forum associated with the software system and/or an area ofendeavor of the software system.
 38. The system for pro-active detectionand correction of low quality questions submitted to a question andanswer based customer support system of claim 35 wherein low qualityquestion formats that are predicted to result in answers that will haveuser satisfaction ratings below a threshold level of user satisfactioninclude: “Who” question formats; “What” question formats; “When”question formats; “Where” question formats; “Why” question formats;“How” question formats; rhetorical question formats; grammaticallyincorrect question formats; and ill-formed question formats.
 39. Thesystem for pro-active detection and correction of low quality questionssubmitted to a question and answer based customer support system ofclaim 35 wherein the one or more corrective actions taken beforeproviding the question data to the one or more support personnelincludes filtering out the improperly formatted question before theimproperly formatted question is provided to the support community, andbefore any resources are devoted to answering the improperly formattedquestion.
 40. The system for pro-active detection and correction of lowquality questions submitted to a question and answer based customersupport system of claim 35 wherein the one or more corrective actionstaken before providing the question data to the one or more supportpersonnel includes refusing to accept submission of the improperlyformatted question before the improperly formatted question is providedto the support community, and before any resources are devoted toanswering the improperly formatted question.
 41. The system forpro-active detection and correction of low quality questions submittedto a question and answer based customer support system of claim 35wherein the one or more corrective actions taken before providing thequestion data to the one or more support personnel includes attemptingto correct the improperly formatted question by providing the user witha set of question transformation instructions used to transform theimproperly formatted question into a properly formatted question. 42.The system for pro-active detection and correction of low qualityquestions submitted to a question and answer based customer supportsystem of claim 41 wherein the user is provided the formattransformation instructions representing suggestions on how tore-phrase/reform the improperly formatted question that are customizedto the specific question data being submitted.
 43. The system forpro-active detection and correction of low quality questions submittedto a question and answer based customer support system of claim 41wherein the user is provided the format transformation instructionsrepresenting suggestions on how to re-phrase/reform the improperlyformatted question that are customized to the specific question databeing submitted as the question data is being submitted.
 44. A systemfor pro-active detection and correction of low quality questionssubmitted to a question and answer based customer support system, thesystem comprising: a software system; a computing system for providingthe software system to one or more users; a question and answer basedcustomer support system through which question data can be entered bythe users, the question data representing questions to potentially beprovided to one or more support personnel associated with the questionand answer based customer support system so that the questionsrepresented by the question data can be answered by at least one of theone or more support personnel; a computing system for providing usersaccess to the question and answer based customer support system and forproviding the users the capability to enter the question datarepresenting questions to potentially be provided to one or more supportpersonnel high quality question format data defining high qualityquestion formats that are predicted to result in answers that will haveuser satisfaction ratings above a threshold level of user satisfaction,defining questions having high quality question formats being defined asproperly formatted questions; a question data receiving module forreceiving question data representing a question submitted by a userthrough the question and answer based customer support system; anquestion format analysis module, the question format analysis moduleanalyzing the question data received at the question data receivingmodule to determine if the question data represents a properly formattedquestion before providing the question data to any of the one or moresupport personnel; and a corrective action module, the corrective actionmodule implementing one or more corrective actions if based on theanalysis of the question data, a determination is made that the questiondata does not represent an improperly formatted question before thequestion data is provided to the one or more support personnel.
 45. Thesystem for pro-active detection and correction of low quality questionssubmitted to a question and answer based customer support system ofclaim 44 wherein the software system is selected from the group ofsoftware systems consisting of: a computing system implemented taxpreparation software system; a network accessed tax preparation softwaresystem; a web-based tax preparation software system; a cloud-based taxpreparation software system; a computing system implemented businessmanagement software system; a network accessed business managementsoftware system; a web-based business management software system; acloud-based business management software system; a computing systemimplemented accounting software system; a network accessed accountingsoftware system; a web-based accounting software system; a cloud-basedaccounting software system; a computing system implemented financialmanagement system; a network accessed financial management system; aweb-based financial management system; and a cloud-based financialmanagement system.
 46. The system for pro-active detection andcorrection of low quality questions submitted to a question and answerbased customer support system of claim 44 wherein the question andanswer based customer support system includes a web-based question andanswer forum associated with the software system and/or an area ofendeavor of the software system.
 47. The system for pro-active detectionand correction of low quality questions submitted to a question andanswer based customer support system of claim 44 wherein high qualityquestion formats that are predicted to result in answers that will haveuser satisfaction ratings above a threshold level of user satisfactioninclude closed-ended question formats.
 48. The system for pro-activedetection and correction of low quality questions submitted to aquestion and answer based customer support system of claim 44 whereinthe one or more corrective actions taken before providing the questiondata to the one or more support personnel includes filtering out thequestion before the question is provided to the support community, andbefore any resources are devoted to answering the question.
 49. Thesystem for pro-active detection and correction of low quality questionssubmitted to a question and answer based customer support system ofclaim 44 wherein the one or more corrective actions taken beforeproviding the question data to the one or more support personnelincludes refusing to accept submission of the question before thequestion is provided to the support community, and before any resourcesare devoted to answering the question.
 50. The method for pro-activedetection and correction of low quality questions submitted to aquestion and answer based customer support system of claim 44 whereinthe one or more corrective actions taken before providing the questiondata to the one or more support personnel includes attempting to correctthe question by providing the user with a set of question transformationinstructions used to transform the question into a properly formattedquestion.
 51. The system for pro-active detection and correction of lowquality questions submitted to a question and answer based customersupport system of claim 50 wherein the user is provided the formattransformation instructions representing suggestions on how tore-phrase/reform the improperly formatted question that are customizedto the specific question data being submitted.
 52. The system forpro-active detection and correction of low quality questions submittedto a question and answer based customer support system of claim 50wherein the user is provided the format transformation instructionsrepresenting suggestions on how to re-phrase/reform the improperlyformatted question that are customized to the specific question databeing submitted while the question data is being submitted.